Loose bolt positioning method and system based on voiceprint feature extraction

Abstract

The invention discloses a loose bolt positioning method and system based on voiceprint feature extraction, and relates to the field of structural health monitoring, and the method comprises the following steps: obtaining a bolt voiceprint signal multi-domain time-frequency feature, and obtaining a voiceprint distillation feature and a voiceprint distillation weight matrix through a feature distillation network; based on the voiceprint distillation weight matrix, carrying out weighted fusion on the voiceprint distillation characteristics to obtain bolt comprehensive characteristics, and comparing pre-obtained fastening bolt comprehensive characteristics to obtain a characteristic similarity matrix; generating a bolt similarity matrix in combination with the obtained physical similarity matrix; and according to the bolt similarity matrix, positioning is carried out in combination with a preset loosening positioning model. The method is used for solving the problems of incomplete characteristic representation and information redundancy in a bolt looseness detection method.

Description

Technical Field

[0001] This invention relates to the field of structural health monitoring, and more specifically, to a method and system for locating loose bolts based on voiceprint feature extraction. Background Technology

[0002] With the rise in transmission voltage levels and the large-scale construction of ultra-high voltage transmission lines, the total number of transmission towers in my country is steadily increasing. Transmission towers made of angle steel are mostly connected by bolts. In the natural environment, due to factors such as wind load vibration, conductor swaying, and temperature changes, the bolted connections may gradually loosen. After a long period of time, this loosening will reduce the structural rigidity of the transmission tower, potentially leading to instability of the tower, affecting the normal operation of the transmission line, and even threatening its safe operation.

[0003] For example, the invention patent announcement CN118503671A discloses a method for locating and maintaining loose tower bolts based on deep learning and voiceprint analysis. This method, belonging to the field of artificial intelligence technology, includes: acquiring current feedback voiceprint data of the tower bolts based on preset sensors; processing the current feedback voiceprint data using a pre-trained tower bolt loosening location model to obtain the current loose tower bolt location; determining the target tower bolt corresponding to the current loosening location; acquiring feedback voiceprint data of the target tower bolt during the overall monitoring period and calculating the structural damage risk coefficient of the tower structure where the target tower bolt is located; and determining a maintenance strategy for the target tower bolt based on the structural damage risk coefficient.

[0004] For example, the invention patent announcement CN119935517A, "Communication Tower Bolt Loosening Location System and Method Based on Voiceprint Feature Extraction," discloses a communication tower bolt loosening location system and method, belonging to the field of tower bolt monitoring. It includes: a sound wave generator fixedly installed on the communication tower and several bone conduction voiceprint sensors. The sound wave generator is used to apply excitation sound wave signals of different frequencies to the communication tower, and the bone conduction voiceprint sensors are used to acquire audio information. It also includes an intelligent analysis terminal for acquiring the audio information sent by the bone conduction voiceprint sensors; the intelligent analysis terminal is used to preprocess the audio information to obtain the target monitoring audio; and a voiceprint recognition module, which is used to perform voiceprint recognition and classification processing on the target monitoring audio, including: the voiceprint recognition module loading the target monitoring audio into a bolt loosening classification model; and performing classification processing based on the bolt loosening classification model to obtain the classification status result of the target monitoring audio; the classification status result includes: normal state, a small number of loose bolts state, and a large number of loose bolts state.

[0005] The above-disclosed technical solutions have at least the following technical problems: The methods for extracting voiceprint features are relatively simple and lack comprehensive modeling of multi-domain signal features, resulting in insufficient robustness of the model in complex noise or nonlinear environments. The acoustic response under different frequency excitations lacks modal correlation analysis, resulting in low model interpretability and difficulty in reflecting the physical and structural characteristics of bolt loosening. Existing models mostly rely on fixed deep learning or classification structures and have not established adaptive learning or cross-domain feature fusion mechanisms, resulting in low utilization efficiency of unlabeled data. The positioning accuracy is limited, and most models can only achieve state recognition at the bolt group or region level, making it difficult to achieve accurate positioning of a single bolt.

[0006] To address the above problems, this invention proposes a solution. Summary of the Invention

[0007] To overcome the aforementioned deficiencies of the prior art, embodiments of the present invention provide a method and system for locating loose bolts based on acoustic signature feature extraction. By constructing an acoustic signature distillation weight matrix, weighting acoustic signature distillation features, introducing physical similarity to obtain a bolt similarity matrix, and processing it to locate loose bolts, the method solves the problems of incomplete feature representation and information redundancy in bolt loosening detection methods.

[0008] To achieve the above objectives, the present invention provides the following technical solution: The loose bolt location method based on acoustic signature feature extraction includes the following steps: acquiring multi-domain time-frequency features of the bolt acoustic signature signal; using a feature distillation network to obtain acoustic signature distillation features and an acoustic signature distillation weight matrix; weighting and fusing the acoustic signature distillation features based on the acoustic signature distillation weight matrix to obtain comprehensive bolt features; comparing these with pre-acquired comprehensive features of fastened bolts to obtain a feature similarity matrix; generating a bolt similarity matrix by combining the obtained physical similarity matrix; and locating the loose bolt based on the bolt similarity matrix and a pre-set loose bolt location model.

[0009] In a preferred embodiment, the process of obtaining the acoustic distillation features and acoustic distillation weight matrix using a feature distillation network specifically involves: performing cross-domain nonlinear mapping on the multi-domain time-frequency features of the bolt acoustic signal to obtain a multi-domain correlation tensor, which is then input into the feature distillation network; the feature distillation network weights the multi-domain correlation tensor using a self-attention mechanism to obtain a weighted acoustic feature representation; using a feedforward neural network to perform layer-by-layer compression and mapping on the weighted acoustic feature representation to generate acoustic distillation features; and based on the activation sequences at each level of the distillation network, constructing an activation feature association graph and generating an acoustic distillation weight matrix through graph structure learning and sparsification processing.

[0010] In a preferred embodiment, the process of generating voiceprint distillation features using a feedforward neural network specifically includes: performing dimensionality pooling on the voiceprint weighted feature representation to obtain a primary representation vector; inputting the primary representation vector into a multi-layer feedforward neural network containing residual connections and layer normalization, and performing feature transformation through layer-by-layer linear transformation and non-linear activation; using the dimensionality reduction mapping layer in the feedforward neural network to compress the feature dimension to the target dimension to form a voiceprint feature compression vector; and performing normalization on the voiceprint feature compression vector to output the voiceprint distillation features.

[0011] In a preferred embodiment, the process of generating a voiceprint distillation weight matrix based on the hierarchical activation sequences in the distillation network specifically includes: performing multi-scale feature extraction on the hierarchical activation sequences to obtain fused features; calculating the contribution weight of each hierarchical activation sequence to the voiceprint distillation features using an attention aggregation module based on the fused features; performing weighted fusion on the hierarchical activation sequences based on the contribution weights to form an intermediate weight feature representation; constructing an activation feature association graph based on the intermediate weight feature representation and generating a voiceprint weight association matrix through graph structure learning; performing sparsification processing on the voiceprint weight association matrix based on a preset association threshold, retaining edge connections with weights higher than the association threshold, and outputting the voiceprint distillation weight matrix.

[0012] In a preferred embodiment, the acoustic distillation weight matrix is ​​used to weight and fuse the acoustic distillation features to obtain bolt composite features. Specifically, the acoustic distillation features are linearly weighted using the acoustic distillation weight matrix to obtain preliminary weighted acoustic features; the preliminary weighted acoustic features are nonlinearly transformed using the Sigmoid activation function to obtain acoustic reconstruction features; the acoustic distillation features and the acoustic reconstruction features are dynamically fused according to preset weights; after normalization, the SelectKBest algorithm is used to select and form bolt composite features.

[0013] In a preferred embodiment, the process of fusing the physical similarity matrix and the feature similarity matrix to generate a bolt similarity matrix through consistency constraints specifically includes: calculating the difference matrix between the feature similarity matrix and the physical similarity matrix; constructing a consistency penalty term based on the difference matrix, wherein the consistency penalty term is positively correlated with the difference; determining the optimal weighting coefficient between the feature similarity matrix and the physical similarity matrix in the fusion process by using an optimization algorithm with the goal of minimizing the consistency penalty term; and performing linear weighted fusion of the feature similarity matrix and the physical similarity matrix based on the optimal weighting coefficient to generate the bolt similarity matrix.

[0014] In a preferred embodiment, the step of locating the bolt based on the bolt similarity matrix and a preset loosening location model includes constructing a Laplace matrix using the bolt similarity matrix. Specifically, this involves: symmetricizing the bolt similarity matrix and truncating small values; using threshold pruning to transform the processed bolt similarity matrix into a sparse adjacency matrix; calculating the degree of each node in the sparse adjacency matrix to construct a diagonal degree matrix; and combining the sparse adjacency matrix and the degree matrix to form the Laplace matrix.

[0015] In a preferred embodiment, the training process of the preset loose localization model specifically includes: dividing the input comprehensive features into labeled and unlabeled data; initializing the weight parameters of the hidden layer based on the extreme learning machine, and calculating the hidden layer output through the activation function; constructing a total loss function based on the supervised loss function and the manifold regularization term, wherein the manifold regularization term is based on the Laplacian matrix; and updating the model's weight parameters by minimizing the total loss function to complete the training.

[0016] In a preferred embodiment, the process of obtaining the time-domain, Mel spectrum, and wavelet packet features of the bolt acoustic signal specifically includes: extracting time-domain features such as zero-crossing rate, root mean square, total energy, and peak amplitude from the waveform of the acoustic signal through framing and windowing; calculating the time-frequency features based on short-time Fourier transform to obtain the Mel spectrum; obtaining the Mel energy through a Mel filter bank and converting it to dB scale; extracting Mel features from the dB-scale Mel energy; performing wavelet packet decomposition on the bolt acoustic signal to obtain the wavelet packet coefficients of each frequency band; and calculating the wavelet packet features for the coefficient sequence of each frequency band.

[0017] The loose bolt location system based on acoustic signature feature extraction includes: a feature distillation module for acquiring multi-domain time-frequency features of bolt acoustic signature signals, and using a feature distillation network to obtain acoustic signature distillation features and an acoustic signature distillation weight matrix; a feature similarity calculation module for weighted fusion of acoustic signature distillation features based on the acoustic signature distillation weight matrix to obtain comprehensive bolt features, and comparing them with pre-acquired comprehensive features of fastened bolts to obtain a feature similarity matrix; a matrix fusion module for generating a bolt similarity matrix by combining the obtained physical similarity matrix; and a model localization module for localization based on the bolt similarity matrix and a preset loose bolt location model.

[0018] The technical effects and advantages of the loose bolt positioning method and system based on voiceprint feature extraction of this invention are as follows: This invention constructs a voiceprint distillation weight matrix through a feature distillation network to perform deep weighted fusion of voiceprint distillation features, achieving dynamic and refined weighting of bolt voiceprint features, significantly improving the discriminative power of the features. Furthermore, it introduces a physical similarity matrix for multi-source similarity joint calculation to obtain a bolt similarity matrix, enhancing the structural consistency and localization reliability of the similarity measurement. A Laplacian matrix is ​​constructed, and through manifold regularization, the learning ability of the semi-supervised extreme learning machine is strengthened, thereby locating loose bolts. This effectively solves the problems of incomplete feature representation and information redundancy in bolt loosening detection methods. Attached Figure Description

[0019] Figure 1 This is a schematic diagram of the loose bolt positioning method based on voiceprint feature extraction provided in an embodiment of the present invention.

[0020] Figure 2 This is a schematic diagram of a loose bolt positioning system based on voiceprint feature extraction, provided in an embodiment of the present invention.

[0021] Figure 3 A comparison chart of different wavelet packet basis classification indices provided for embodiments of the present invention.

[0022] Figure 4 The diagram illustrates the effect of different K values ​​on the results, as provided in this embodiment of the invention.

[0023] Figure 5 This is the confusion matrix of the output result of the semi-supervised extreme learning machine provided in the embodiments of the present invention.

[0024] Figure 6 This is a distribution diagram of the bolt loosening acoustic signal acquisition points provided in an embodiment of the present invention. Detailed Implementation

[0025] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0026] Example 1, Figure 1 The present invention provides a method for locating loose bolts based on voiceprint feature extraction, comprising the following steps: S1, obtain the multi-domain time-frequency features of the bolt acoustic signature signal, and use the feature distillation network to obtain the acoustic signature distillation features and acoustic signature distillation weight matrix; S2, based on the acoustic distillation weight matrix, the acoustic distillation features are weighted and fused to obtain the bolt comprehensive features, and the feature similarity matrix is ​​obtained by comparing with the pre-acquired fastening bolt comprehensive features; S3, combine the obtained physical similarity matrix to generate the bolt similarity matrix; S4. Locate the bolt based on the bolt similarity matrix and the preset loosening positioning model.

[0027] S1: Obtain the multi-domain time-frequency features of the bolt acoustic signature signal, and use the feature distillation network to obtain the acoustic signature distillation features and acoustic signature distillation weight matrix.

[0028] In this embodiment, the process of obtaining the voiceprint distillation features and voiceprint distillation weight matrix using a feature distillation network is specifically as follows: A multi-domain correlation tensor is obtained by performing a cross-domain nonlinear mapping on the multi-domain time-frequency features of the bolt acoustic signature signal, and then input into the feature distillation network. The feature distillation network uses a self-attention mechanism to weight multi-domain correlation tensors to obtain a weighted feature representation of the voiceprint. By using a feedforward neural network to compress and map the weighted feature representation of voiceprint layer by layer, voiceprint distillation features are generated. Based on the activation sequences of each level in the distillation network, an activation feature association graph is constructed, and a voiceprint distillation weight matrix is ​​generated through graph structure learning and sparsification.

[0029] It should be noted that in the process of obtaining multi-domain correlation tensors through cross-domain nonlinear mapping, it is necessary to first construct a cross-domain feature encoder, which contains three parallel feature processing channels, used to process time-domain feature sequences, Mel spectrum features, and wavelet packet feature vectors, respectively.

[0030] The three parallel feature processing channels each consist of several layers of one-dimensional convolutional modules. The temporal feature channel uses a narrow convolutional kernel to extract local temporal patterns, the Mel spectrum channel uses a wide convolutional kernel to capture frequency domain structural features, and the wavelet packet feature channel performs feature transformation through a fully connected layer.

[0031] Furthermore, the feature maps (time domain, frequency domain, and wavelet packet features) obtained from the three channels are dimensionally aligned using zero-padding to ensure that the output feature dimensions of different channels are consistent, thus aligning the feature maps output by each channel to the same dimension.

[0032] In this embodiment, the weighting of multi-domain correlation tensors through a self-attention mechanism to obtain a weighted voiceprint feature representation is specifically as follows: Using a trainable weight matrix, a linear transformation is performed on a multi-domain feature tensor to obtain a query matrix, a key matrix, and a value matrix; Attention weights are obtained by calculating the correlation between the query matrix and the key matrix through the inner product; Based on attention weights, the value matrix is ​​further weighted to output a weighted feature representation of the voiceprint.

[0033] In this embodiment, the process of generating voiceprint distillation features using a feedforward neural network specifically includes: The weighted feature representation of the voiceprint is subjected to dimensionality pooling to obtain the primary representation vector; The primary representation vector is input into a multi-layer feedforward neural network containing residual connections and layer normalization, and feature transformation is performed through layer-by-layer linear transformation and non-linear activation. By utilizing the dimension reduction mapping layer in the feedforward neural network, the feature dimension is compressed to the target dimension, forming a voiceprint feature compression vector; The voiceprint feature compression vector is normalized to output the voiceprint distillation feature.

[0034] It should be noted that the target dimension needs to be consistent with the number of features selected by the SelectKBest algorithm in weight 5. It is related to the number of features selected by the SelectKBest algorithm. After the SelectKBest algorithm determines the optimal value of K, the number of output neurons of the dimension reduction mapping layer is set to this value to ensure that the generated voiceprint distillation features and the comprehensive features have the same dimensional space. This allows the features based on statistical screening and the features based on feature distillation to be used for subsequent similarity calculations and matrix operations on the same dimension.

[0035] In this embodiment, the process of generating the voiceprint distillation weight matrix based on the hierarchical activation sequence in the distillation network is specifically as follows: Multi-scale feature extraction is performed on the hierarchical activation sequence to obtain fused features; Based on the fusion features, the contribution weight of each level of activation sequence to the voiceprint distillation features is calculated through the attention aggregation module. The hierarchical activation sequences are weighted and fused based on contribution weights to form intermediate weight feature representations; An activation feature association graph is constructed based on intermediate weight feature representation, and a voiceprint weight association matrix is ​​generated through graph structure learning. The voiceprint weight association matrix is ​​sparsified based on a preset association threshold, and edge connections with weights higher than the association threshold are retained to output the voiceprint distillation weight matrix.

[0036] It should be noted that multi-scale feature extraction to obtain fused features involves performing convolution operations on each layer activation using multiple convolution kernels to obtain feature representations at different scales, and then concatenating the features at multiple scales to form fused features.

[0037] Furthermore, the fused features are input into the attention aggregation module, and the contribution weight of each activation sequence is calculated using the attention weight calculation formula.

[0038] The formula for calculating the attention weight is as follows: , In the formula, This represents the contribution weight value of the fused features. For querying the matrix, This is the transpose of the key matrix. For value matrices, denoted as the dimension of the key matrix.

[0039] The contribution weights are used to weight and fuse the hierarchical activation sequences, and the calculation formula is as follows: , In the formula, This represents the intermediate weight feature, where L is the total number of levels. The convolutional features of the l-th layer are represented by these features.

[0040] Furthermore, the intermediate weighted feature representation is used as node input to construct an activation feature association graph. The nodes represent features, and the edge connections of the activation feature association graph are obtained by calculating the similarity of each feature in the weighted feature representation.

[0041] It should be noted that sparsification is a process that sets an association threshold and sparsifies the edge connections in the voiceprint weight association matrix, retaining only connections with weights greater than the association threshold.

[0042] S2, based on the acoustic distillation weight matrix, the acoustic distillation features are weighted and fused to obtain the bolt comprehensive features, and the feature similarity matrix is ​​obtained by comparing with the pre-acquired fastening bolt comprehensive features.

[0043] In this embodiment, the acoustic distillation weight matrix weights and fuses the acoustic distillation features to obtain the bolt comprehensive features, specifically: Preliminary weighted voiceprint features are obtained by linearly weighting the voiceprint distillation features using the voiceprint distillation weight matrix. The voiceprint reconstruction features are obtained by performing a nonlinear transformation on the preliminary weighted voiceprint features using the Sigmoid activation function. The voiceprint distillation features and voiceprint reconstruction features are dynamically fused according to preset weights; After normalization, the SelectKBest algorithm is used to select and form the bolt composite features.

[0044] It should be noted that by using multi-level weighting, nonlinear transformation and fusion operations, the transformation process from acoustic distillation features to bolt comprehensive features is gradually optimized, thereby gradually improving the expressive and discriminative capabilities of the features and further enhancing the accuracy and robustness of the loose bolt positioning system.

[0045] It should be noted that the SelectKBest algorithm calculates the correlation score between each feature and the bolt loosening state. The score is determined based on statistical verification methods. All features are sorted according to the correlation score, and the K features with the highest correlation are selected from the sorting results to form a comprehensive feature.

[0046] like Figure 4 As shown in the experimental results, when the value of K ranges from 30 to 90, the classification accuracy fluctuates as K increases. The highest accuracy occurs when the K value is between 40 and 60. Within this range, the accuracy remains relatively high and stable. When choosing the K value, selecting too many features (too large a K value) may lead to overfitting, while selecting too few features (too small a K value) may lead to underfitting. A K value between 40 and 60 can strike a good balance between the two.

[0047] In this embodiment, the process of obtaining the feature similarity matrix by comparing the pre-acquired comprehensive features of the fastening bolts is specifically as follows: The comprehensive features of the bolt to be detected are matched with the comprehensive features of the pre-acquired fastening bolts, and the similarity between them is calculated. Cosine similarity is calculated for each pair of bolt features to obtain the evaluated similarity value between each bolt to be detected and the fastening bolt. A feature similarity matrix is ​​constructed based on the evaluated similarity values. Each element in the feature similarity matrix represents the degree of similarity between the bolt to be detected and the fastening bolt.

[0048] S3, combine the obtained physical similarity matrix to generate the bolt similarity matrix.

[0049] In this embodiment, the process of fusing the physical similarity matrix and the feature similarity matrix through consistency constraints to generate the bolt similarity matrix is ​​specifically as follows: Calculate the difference matrix between the feature similarity matrix and the physical similarity matrix; A consistency penalty term is constructed based on the dissimilarity matrix, and the consistency penalty term is positively correlated with the dissimilarity. With the goal of minimizing the consistency penalty term, the optimal weighting coefficients of the feature similarity matrix and the physical similarity matrix in the fusion process are determined by an optimization algorithm; Based on the optimal weighting coefficients, the feature similarity matrix and the physical similarity matrix are linearly weighted and fused to generate the bolt similarity matrix.

[0050] In this embodiment, the process of obtaining the physical similarity matrix is ​​as follows: Based on the engineering design drawings of the iron tower, obtain the spatial coordinate information and connection relationship of each bolt in the iron tower; Based on the connection relationships and distance constraints of bolts in the iron tower, the structural adjacency matrix of the bolts is constructed; The physical similarity matrix is ​​calculated based on the structural adjacency matrix.

[0051] It should be noted that distance constraint refers to adjusting the adjacency matrix by calculating the physical distance between bolts using the Euclidean distance formula in three-dimensional spatial coordinates.

[0052] The physical similarity is specifically calculated using the following formula: , In the formula, For physical similarity, These are the corresponding elements in the structural adjacency matrix. The preset bandwidth parameter of the Gaussian kernel function. Let be the Euclidean distance between bolt i and bolt j.

[0053] It should be noted that the difference matrix is ​​calculated by subtracting the feature similarity matrix from the physical similarity matrix element by element.

[0054] Furthermore, based on the dissimilarity matrix, a consistency penalty term is constructed. This penalty term measures the inconsistency between the physical similarity matrix and the recalibrated feature similarity matrix. The consistency penalty term is proportional to the sum of the elements of the dissimilarity matrix.

[0055] It should be noted that the objective is to minimize the consistency penalty term. The optimal weighting coefficients are dynamically determined through gradient descent and used to weight the feature similarity matrix and the physical similarity matrix, respectively. The optimal weighting coefficients ensure that the fused similarity matrix minimizes the consistency penalty term while maintaining the complementarity of the two matrices as much as possible.

[0056] S4. Locate the bolt based on the bolt similarity matrix and the preset loosening positioning model.

[0057] In this embodiment, the positioning based on the bolt similarity matrix and a preset loosening positioning model includes constructing a Laplace matrix using the bolt similarity matrix, specifically: The bolt similarity matrix is ​​symmetricized and truncated to its smallest value. Threshold pruning is used to transform the processed bolt similarity matrix into a sparse adjacency matrix; Calculate the degree of each node in the sparse adjacency matrix and construct a diagonal degree matrix; By combining the sparse adjacency matrix and the degree matrix, a Laplace matrix is ​​formed.

[0058] It should be noted that the pruning threshold is set based on the distribution of the bolt similarity matrix. The threshold is selected by analyzing the statistical characteristics of the bolt similarity matrix. For each pair of nodes in the bolt similarity matrix, if the similarity between them is greater than the set threshold, the corresponding edge connection is retained in the sparse adjacency matrix.

[0059] Furthermore, through pruning, the resulting sparse adjacency matrix is ​​a symmetric matrix where the elements represent the similarity between nodes. Most elements are zero, and only strongly connected node pairs are retained. Therefore, this adjacency matrix is ​​a sparse matrix, which means that only non-zero elements need to be stored during storage to improve computational efficiency and save storage space.

[0060] In this embodiment, the training process of the preset loosening positioning model is specifically as follows: The comprehensive features of the input are divided into labeled and unlabeled data; The weight parameters of the hidden layer are initialized based on the Extreme Learning Machine, and the output of the hidden layer is calculated through the activation function. A total loss function is constructed based on the supervised loss function and the popularization regularization term, wherein the popularization regularization term is based on the Laplacian matrix; Training is completed by minimizing the total loss function and updating the model's weight parameters.

[0061] In this embodiment, the acquisition of the supervised loss function is specifically as follows: Prediction results obtained through forward propagation of the model; Calculate the error between the predicted value and the actual value of the labeled data; The supervised loss function is obtained by taking the average of all errors.

[0062] The specific calculation formula for the supervised loss function is as follows: , In the formula, For the supervision loss function, and Let be the predicted value and the actual value of the i-th sample, respectively. This represents the number of labeled samples.

[0063] It should be noted that the popular regularization term constrains the nodal feature representation of bolts through the Laplacian matrix, which is used to ensure that bolt nodes that are similar in the bolt similarity matrix have similar nodal feature representations in the model output.

[0064] The specific formula for calculating the popular regularization term is as follows: , In the formula, For popular regularization terms, The total number of samples, For each element in the Laplace matrix, Let be the feature representation of the i-th node. Let be the feature representation of the j-th node.

[0065] The total loss function is calculated using the following formula: , In the formula, For the total loss function, The weights of the predefined regularization terms are used to control the balance between the supervised loss and the popular regularization terms.

[0066] It should be noted that in a semi-supervised extreme learning machine model, the output matrix of the hidden layer represents the node feature representation of each node in the hidden layer.

[0067] In this embodiment, the process of the trained model outputting the location identifier of the loose bolt is as follows: The comprehensive feature subset is input into the trained and optimized semi-supervised extreme learning machine model, and the model calculates the output layer response vector through the hidden layer nonlinear mapping. Calculate the loosening probability of each bolt based on the magnitude of each response value in the output layer response vector; The probability of loosening is compared with a preset judgment threshold to determine the loosening status; Then, based on the location index of the maximum response value, output the location identifier corresponding to the bolt.

[0068] like Figure 5 As shown, combined with Figure 6 By printing the confusion matrix of the classification results of the semi-supervised extreme learning machine model when the proportion of labeled data is 0.1, it can be found that the model can correctly locate most categories with a high accuracy. The classification of tight (label 10) and loose also achieves a high accuracy. However, there is a lot of confusion between labels 5 (connection 1 of D column), 6 (connection 2 of D column), and 8 (connection 4 of D column), indicating that the characteristics of the bolt loosening positions represented by the labels are quite similar, making it difficult for the model to distinguish the voiceprint signals corresponding to these labels.

[0069] In this embodiment, the process of obtaining the time-domain, Mel-spectrum, and wavelet packet features of the bolt acoustic signature signal is as follows: Time-domain features such as zero-crossing rate, root mean square, total energy, and peak amplitude are extracted from the waveform of the acoustic signal through frame segmentation and windowing. The Mel spectrum is obtained by calculating the time-frequency features based on the short-time Fourier transform, the Mel energy is obtained by the Mel filter bank, and then converted to the dB scale. The Mel features are extracted from the dB scale Mel energy. Wavelet packet decomposition is performed on the bolt acoustic signature signal to obtain the wavelet packet coefficients of each frequency band, and wavelet packet features are obtained by calculating the coefficient sequence of each frequency band.

[0070] It should be noted that the time-domain features include zero-crossing rate, root mean square, total energy, and peak amplitude; the Mel features include mean, standard deviation, kurtosis, and skewness; the Mel features include the mean, standard deviation, kurtosis, and skewness of the energy sequence corresponding to each Mel frequency band in the Mel spectrogram; and the wavelet packet features include the mean, standard deviation, kurtosis, and skewness of the coefficient sequence of each wavelet packet frequency band.

[0071] In this embodiment, during the selection of wavelet packet basis in wavelet packet transform, the influence of different wavelet packet basis on the experimental results is explored through experiments, and finally the coif3 wavelet packet basis is selected.

[0072] It should be noted that before acquiring the acoustic signal, the bolts are first marked with their numbers according to a preset rule, specifically: The sound-generating device and the sound-collecting device are placed diagonally; Starting from the main material where the sound-generating device is installed, mark the main materials counterclockwise; Then number the connection points from bottom to top; Each bolt is marked with a main material-connection structure location identifier.

[0073] In this embodiment, the calculation of the Mel spectrum of the acoustic signal and its conversion to a dB scale specifically involves: By using a Mel filter, 20 center frequencies are uniformly selected on the Mel scale; Based on the conversion formula between Mel frequency and Hertz, linear frequency is converted into Mel frequency; Create a triangular filter bank, with each filter evenly distributed on the Mel scale; The energy of each Mel band is obtained by inputting the linear power spectrum; The energy of the Mel band is logarithmically converted to the dB scale.

[0074] It should be noted that because the range of audio energy variation is very large, converting it to the dB scale can effectively compress the dynamic range of energy variation, preventing the value from being too large or too small.

[0075] like Figure 3 As shown, five common wavelet envelope bases—db4 (Daubechies wavelet), coif3 (Coiflet wavelet), sym5 (Symlets wavelet), bior3.3 (Biorthogonal wavelet), and haar (Haar wavelet)—were selected for experiments to investigate the influence of wavelet envelope bases on the results. By comparing the classification accuracy of five wavelet envelope bases in the context of a label scaling factor of 0.1 and various classification metrics for easily confused classes, it can be seen that different wavelet envelope bases show significant differences in their classification performance for category 5. db4 has an accuracy of 0.88, with other metrics being balanced and its overall performance being good. coif3 performs best among all metrics, with an accuracy close to 0.95, and its precision, recall, and F1 score are also higher than other wavelet bases. sym5 performs second best, especially in terms of relatively high accuracy, but its precision and recall are slightly lower. bior3.3 has a high accuracy, but its performance in other metrics is slightly lower. In contrast, haar has low performance in all metrics, and its F1 score fluctuates greatly. Based on the experimental data, coif3 is the best wavelet envelope base for processing audio signals for bolt loosening detection, which helps improve the robustness and accuracy of the classifier.

[0076] It should be noted that although the formulas for calculating the mean, standard deviation, kurtosis, and skewness are consistent in both the Mel spectral domain and the wavelet packet domain, the data they process have different physical meanings and time-frequency characteristics. Mel spectral features focus on describing the frequency distribution and energy changes of the signal, making them suitable for obtaining the continuous vibration modes of loose bolts; while wavelet packet features are more suitable for analyzing instantaneous changes and impacts in the signal, and can effectively detect sudden vibrations and impacts of loose bolts. By extracting features from these two different domains, the time-frequency characteristics of loose bolts can be comprehensively reflected.

[0077] It should be noted that the mean characteristic of the wavelet packet domain reflects the overall energy level of the acoustic signature signal in a specific frequency band, and can describe the long-term variation trend of the acoustic signature signal in that frequency band. It is suitable for obtaining the continuous vibration mode of a loose bolt in a certain frequency band.

[0078] The mean of the wavelet packet frequency band coefficient sequence is calculated using the following formula: , In the formula, Let be the mean of the frequency band coefficient sequence of the r-th wavelet packet in the l-th layer, T be the total number of data points, and t be the time point. This represents the frequency band of the r-th wavelet packet in the l-th layer.

[0079] It should be noted that the standard deviation characteristic of the wavelet packet domain represents the variability of the signal within the frequency band. A larger standard deviation usually indicates a larger amplitude of signal variation within the frequency band. For a loose bolt, a larger standard deviation may indicate the presence of strong vibration or a sudden impact event in the signal.

[0080] The standard deviation of the wavelet packet frequency band coefficient sequence is specifically calculated using the following formula: , In the formula, denoted as the standard deviation of the frequency band coefficient sequence of the r-th wavelet packet in the l-th layer.

[0081] It should be noted that the kurtosis feature of the wavelet packet domain is used to obtain sharp changes or sudden high-amplitude components of the voiceprint signal. The impact changes of loose bolts often show high kurtosis in certain frequency bands, thus reflecting the high amplitude fluctuations of the bolt when it collides with other objects or is subjected to force.

[0082] The peak value of the wavelet packet frequency band coefficient sequence is specifically calculated using the following formula: , In the formula, The peak value of the frequency band coefficient sequence of the r-th wavelet packet in the l-th layer is given. This is to standardize the kurtosis so that the kurtosis of the normal distribution is 0.

[0083] It should be noted that the skewness characteristic of the wavelet packet domain indicates the skewness of the voiceprint signal distribution, reflecting whether the energy distribution of the voiceprint signal in certain frequency bands tends to be high-frequency or low-frequency. A frequency band with a larger skewness value may mean that the bolt has generated a vibration or impact in a more biased direction in that frequency band.

[0084] The skewness of the wavelet packet frequency band coefficient sequence is specifically calculated using the following formula: , In the formula, denoted as the skewness value of the frequency band coefficient sequence of the r-th wavelet packet in the l-th layer, where a positive value indicates a right skewness and a negative value indicates a left skewness.

[0085] Example 2, Figure 2 The present invention provides a loose bolt positioning system based on voiceprint feature extraction, comprising: The feature distillation module is used to obtain the multi-domain time-frequency features of the bolt acoustic signature signal. The feature distillation network is used to obtain the acoustic signature distillation features and the acoustic signature distillation weight matrix. The feature similarity calculation module uses the acoustic distillation weight matrix to weight and fuse the acoustic distillation features to obtain the bolt comprehensive features, and compares them with the pre-acquired fastening bolt comprehensive features to obtain the feature similarity matrix. The matrix fusion module is used to combine the obtained physical similarity matrix to generate a bolt similarity matrix; The model positioning module is used to locate the bolts based on the bolt similarity matrix and a preset loosening positioning model.

[0086] The above formulas are all dimensionless calculations. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters in the formulas are set by those skilled in the art according to the actual situation.

[0087] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, in the form of a computer program product.

[0088] Those skilled in the art will recognize that the modules and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0089] In addition, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module.

[0090] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0091] In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A loose bolt positioning method based on voiceprint feature extraction, characterized in that, The method comprises the following steps: obtaining bolt voiceprint signal multi-domain time-frequency features, using a feature distillation network to obtain voiceprint distillation features and a voiceprint distillation weight matrix; weighting and fusing the voiceprint distillation features based on the voiceprint distillation weight matrix to obtain bolt comprehensive features, and comparing the bolt comprehensive features with pre-obtained fastening bolt comprehensive features to obtain a feature similarity matrix; combining the obtained physical similarity matrix to generate a bolt similarity matrix; positioning based on the bolt similarity matrix and a pre-set loosening positioning model.

2. The loose bolt positioning method based on voiceprint feature extraction according to claim 1, characterized in that, The process of using the feature distillation network to obtain the voiceprint distillation features and the voiceprint distillation weight matrix comprises the following steps: cross-domain nonlinear mapping is performed on the multi-domain time-frequency features of the bolt voiceprint signal to obtain a multi-domain correlation tensor, which is input into the feature distillation network; the feature distillation network weights the multi-domain correlation tensor through a self-attention mechanism to obtain a voiceprint weighted feature representation; a feedforward neural network is used to compress and map the voiceprint weighted feature representation layer by layer to generate voiceprint distillation features; based on the activation sequences of each level in the distillation network, an activation feature correlation graph is constructed, and a voiceprint distillation weight matrix is generated through graph structure learning and sparsification processing.

3. The loose bolt positioning method based on voiceprint feature extraction according to claim 2, characterized in that, The process of using the feedforward neural network to generate the voiceprint distillation features comprises the following steps: dimensional pooling processing is performed on the voiceprint weighted feature representation to obtain a primary feature vector; the primary feature vector is input into a multi-layer feedforward neural network comprising residual connections and layer normalization, and feature transformation is performed through layer-by-layer linear transformation and nonlinear activation; a dimension reduction mapping layer in the feedforward neural network is used to compress the feature dimension to a target dimension to form a voiceprint feature compression vector; normalization processing is performed on the voiceprint feature compression vector to output the voiceprint distillation features.

4. The loose bolt positioning method based on voiceprint feature extraction according to claim 3, characterized in that, The process of generating the voiceprint distillation weight matrix based on the level activation sequences in the distillation network comprises the following steps: multi-scale feature extraction is performed on the level activation sequences to obtain fusion features; based on the fusion features, the contribution weight of each level activation sequence to the voiceprint distillation features is calculated through an attention aggregation module; the level activation sequences are weighted and fused based on the contribution weight to form an intermediate weight feature representation; an activation feature correlation graph is constructed based on the intermediate weight feature representation, and a voiceprint weight correlation matrix is generated through graph structure learning; sparsification processing is performed on the voiceprint weight correlation matrix based on a pre-set correlation threshold to retain edge connections with a weight higher than the correlation threshold, and a voiceprint distillation weight matrix is output.

5. The loose bolt positioning method based on voiceprint feature extraction according to claim 4, characterized in that, The voiceprint distillation weight matrix is used to weight and fuse the voiceprint distillation features to obtain bolt comprehensive features, which comprises the following steps: the voiceprint distillation features are linearly weighted using the voiceprint distillation weight matrix to obtain preliminary weighted voiceprint features; nonlinear transformation is performed on the preliminary weighted voiceprint features through a Sigmoid activation function to obtain voiceprint reconstruction features; the voiceprint distillation features and the voiceprint reconstruction features are dynamically fused according to a pre-set weight; after normalization processing, the SelectKBest algorithm is used to select and form bolt comprehensive features.

6. The loose bolt positioning method based on voiceprint feature extraction according to claim 5, characterized in that, The process of fusing the physical similarity matrix and the feature similarity matrix to generate a bolt similarity matrix through consistency constraint comprises the following steps: a difference matrix between the feature similarity matrix and the physical similarity matrix is calculated; A consistency penalty term is constructed based on the difference matrix, and the consistency penalty term is positively correlated with the difference degree; An optimization algorithm is used to determine the optimal weighting coefficient of the feature similarity matrix and the physical similarity matrix in the fusion process by minimizing the consistency penalty term; Based on the optimal weighting coefficient, the feature similarity matrix and the physical similarity matrix are linearly weighted and fused to generate a bolt similarity matrix.

7. The loose bolt positioning method based on voiceprint feature extraction according to claim 6, characterized in that, The bolt similarity matrix is combined with a preset loose positioning model for positioning, including constructing a Laplacian matrix using the bolt similarity matrix, specifically: The bolt similarity matrix is symmetrically processed and small values are truncated; The processed bolt similarity matrix is converted into a sparse adjacency matrix by threshold pruning; The degree of each node in the sparse adjacency matrix is calculated to construct a diagonal degree matrix; The sparse adjacency matrix and the degree matrix are combined to form a Laplacian matrix.

8. The loose bolt positioning method based on voiceprint feature extraction according to claim 7, characterized in that, The training process of the preset loose positioning model is specifically as follows: The input comprehensive features are divided into labeled and unlabeled data; The weight parameters of the hidden layer are initialized based on the extreme learning machine, and the hidden layer output is calculated by the activation function; A total loss function is constructed based on the supervised loss function and the popular regularization term, and the popular regularization term is based on the Laplacian matrix; The weight parameters of the model are updated by minimizing the total loss function to complete the training.

9. The loose bolt positioning method based on voiceprint feature extraction according to claim 8, characterized in that, The time domain, mel spectrum and wavelet packet feature acquisition process of the bolt voiceprint signal is specifically as follows: Time domain features such as zero-crossing rate, root mean square, total energy and peak amplitude are extracted from the waveform of the acoustic signal through framing and windowing processing; Mel spectrum is obtained by calculating time-frequency features based on short-time Fourier transform, and mel energy is obtained by mel filter bank, and is converted to dB scale, and mel features are extracted from the mel energy of the dB scale; Wavelet packet decomposition is performed on the bolt voiceprint signal to obtain wavelet packet coefficients of each frequency band, and wavelet packet features are obtained by calculating the coefficient sequence of each frequency band.

10. A system using the loose bolt positioning method based on voiceprint feature extraction according to any one of claims 1-9, comprising: A feature distillation module is used to acquire multi-domain time-frequency features of the bolt voiceprint signal, and a voiceprint distillation feature and a voiceprint distillation weight matrix are obtained by using a feature distillation network; A feature similarity calculation module is used to weight and fuse the voiceprint distillation features based on the voiceprint distillation weight matrix to obtain a bolt comprehensive feature, and a feature similarity matrix is obtained by comparing the pre-acquired fastening bolt comprehensive feature; A matrix fusion module is used to generate a bolt similarity matrix in combination with the obtained physical similarity matrix; A model positioning module is used to position the bolt similarity matrix in combination with a preset loose positioning model.

Images