A bolt loosening monitoring method based on small sample learning sound recognition and a portable detection device

Abstract

The application provides a bolt loosening monitoring method based on small sample learning sound recognition and a portable detection device, adopts an improved small sample learning model suitable for a sound signal and a sound signal data enhancement method based on time domain / frequency domain masking, compared with directly using a small sample learning model in an image recognition field, the improved model can obtain higher bolt loosening recognition precision, and the bolt loosening recognition precision under a small amount of samples is further improved through an example verification; in addition, the training sample is enhanced by using the proposed data enhancement method, the data enhancement method is more suitable for the sound signal, and the bolt loosening recognition precision under a small amount of samples is further improved, and finally, the model completed through training is integrated into an embedded board card, so that the portable bolt loosening detection device is realized.

Description

Technical Field

[0001] This invention relates to the field of structural health monitoring, specifically to a bolt loosening monitoring method and portable detection device based on small-sample learning sound recognition. Background Technology

[0002] Bolts are widely used in structures for connection and load transfer. In the aerospace field, for example, an aircraft uses a large number of bolts. Although bolted connections are quite secure, they can still loosen under various complex load impacts. The hazards caused by loose bolts are significant; for instance, the train derailment in Lamborgh, UK in February 2007 was caused by loose wheel nuts. Therefore, monitoring bolt loosening is of great importance.

[0003] Currently, there is considerable research on bolt loosening detection in various structural systems using structural health monitoring (SHM) methods. Commonly used techniques include vibration-based, electromechanical impedance-based, and guided wave-based SHM technologies. Since assembled structures typically consist of many bolts and bolted connections are local structural elements, bolt loosening at a local location does not significantly alter the overall structural dynamics. Therefore, vibration-based SHM is not sensitive to changes in bolt preload, resulting in poor monitoring sensitivity. Electromechanical impedance-based SHM has a detection range limited to the vicinity of the piezoelectric sensor and requires expensive, high-precision impedance analyzers, thus presenting limitations. Ultrasonic guided wave-based SHM still requires attaching piezoelectric sensors, hindering its application. Some scholars have proposed using image processing for bolt loosening detection (see patent, Cui Chuang, Lao Wulue, Zhang Qinghua, et al. A bolt loosening detection method, system, and storage medium based on deep learning [P]. Sichuan Province: CN114742820B, 2023-06-27.), but this method requires recording torque references under standard torque, making it inconvenient for application.

[0004] Sound signals typically have a higher frequency band than vibration signals and do not require excitation from piezoelectric ceramic sensors, exhibiting high sensitivity to damage and becoming a research hotspot in the field of structural damage monitoring. In 2020, Professors Furui Wang and Gangbing Song of the University of Houston proposed using a "percussion" method to monitor bolt loosening. This method converts speech signals into two-dimensional Mel spectrograms (MFCCs) and establishes a convolutional bidirectional long short-term memory neural network to identify the Mel spectrograms (see the literature Wang F, Song G. Looseness detection in cup-lock scaffolds using percussion-based method[J]. Automation in Construction, 2020, 118: 103266.). However, the Mel spectrograms used in this method are two-dimensional data, and the deep learning model it establishes is large and the data volume is too large, making it difficult to apply in embedded hardware and unable to perform portable online detection in actual structures. Domestic scholars (see patent, Shi Xiaowei, Yue Shichao. A bolt failure detection method [P]. Jiangsu Province: CN113804767B, 2022-11-04.) have adopted similar methods. Some scholars (see literature Zhuo Debing, Cao Hui. Bolt connection damage identification based on wavelet time-frequency diagram and lightweight convolutional neural network [J]. Engineering Mechanics, 2021, 38(09):228-238.) have converted the sound signal into a wavelet time-frequency diagram as input and used the convolutional neural network MobileNetv2 for training and recognition, but it still depends on large data acquisition equipment and requires a large amount of training data. Some scholars have used Long Short-Term Memory Network (LSTM) to classify sound signals (see patent, Lu Wei, Ma Hua, Jiang Xingmeng. An acoustic-based rail bolt loosening detection device [P]. Henan Province: CN115127797A, 2022-09-30.), but this method also requires a large amount of labeled data for training the LSTM model.

[0005] It can be seen that sound signals have high sensitivity to structural damage, enabling the detection of loose bolts without the need for attached sensors. However, existing percussion methods for processing sound signals are quite complex, relying on large data acquisition and processing devices, making it difficult to achieve portable detection of loose bolts in engineering structures. Furthermore, the training of deep networks heavily depends on a large amount of labeled data, but in practical applications, it is difficult to obtain a large number of labeled samples. Summary of the Invention

[0006] To address the problems mentioned in the background section, few-shot learning and data augmentation methods can be employed in portable bolt loosening monitoring based on sound recognition. Few-shot learning refers to the ability of previously learned categories to help predict new categories when only a few labeled samples are available. This technique effectively reduces the amount of data required to train deep network models. Simultaneously, data augmentation methods can increase the number of samples, thereby improving network accuracy. However, currently, few-shot learning and data augmentation methods are mainly used in image recognition, and their effectiveness in directly applying them to sound signals is not ideal. Therefore, this invention proposes a bolt loosening monitoring method based on few-shot learning sound recognition, employing an improved few-shot learning model suitable for sound signals and a sound signal data augmentation method based on time-domain / frequency-domain masking. Compared to directly using a few-shot learning model from the image recognition field, the improved model achieves higher bolt loosening detection accuracy, as verified by computational examples. Furthermore, the proposed data augmentation method is used to enhance the training samples; this data augmentation method is more suitable for sound signals and can further improve the accuracy of bolt loosening detection with a small number of samples. Finally, the trained model is integrated into an embedded board to realize a portable bolt loosening detection device.

[0007] The technical solution of this invention is as follows:

[0008] A bolt loosening monitoring method based on few-shot learning sound recognition includes the following steps:

[0009] Step 1: Build a few-shot learning model; the few-shot learning model includes an embedding function and a metric module; the embedding function is implemented using a deep learning neural network, and the metric module is implemented using a cosine distance function;

[0010] Step 2: Using the bolted connection structure actually installed on the overall structure as the target domain structure, the target domain structure is struck under different bolt torque states, and the sound data generated by the striking is collected; the collected sound data is preprocessed to obtain the target domain dataset, and the target domain dataset is divided into a support set S and a test set T, wherein the support set S contains training samples with category labels, and the category labels refer to the bolt torque state;

[0011] Step 3: Prepare an independent source domain structure identical to the target domain structure. Tap the source domain structure under different bolt torque conditions and collect the resulting sound data. Preprocess the collected sound data to obtain an auxiliary set A. Enhance the auxiliary set A using a data augmentation method based on time-domain and frequency-domain masking to obtain the enhanced auxiliary set A1. Specifically: during time-domain masking data augmentation, randomly set all values ​​of the collected sound signal for a certain time period to zero, using this as new data. In frequency masking, perform a Fourier transform on the collected sound signal, then select a frequency range outside the center frequency band within the entire frequency band of the sound signal for masking, and finally perform an inverse Fourier transform on the masked signal to obtain new data.

[0012] Step 4: Train the model on the auxiliary set A1 for multiple epochs;

[0013] During a training cycle, multiple auxiliary support subsets A are randomly constructed from the auxiliary set A1. s and auxiliary test subset A t Auxiliary support subset A s The categories and number of samples in the auxiliary test subset A are the same as those in the support set S. t The categories and number of samples in set A are the same as those in the test set T; a set A s and A t This constitutes an episode, and the few-shot learning model built in step 1 is trained sequentially using multiple constructed episodes;

[0014] A is reconstructed randomly in each training cycle. s and A t Then, the model trained in the previous cycle is used for further training; at the end of the last training cycle, the trained small sample learning model is saved for bolt loosening monitoring of the target domain structure.

[0015] Step 5: Input the training samples from the support set S into the embedding function of the few-shot learning model trained in Step 4, calculate the embedding vector of each training sample, and average the embedding vectors of all samples of the same class in the support set S to obtain the prototype vector of that class; input the samples from the test set T into the embedding function of the few-shot learning model trained in Step 4, calculate the embedding vector of each test sample, calculate the cosine distance between the embedding vector of each test sample and the prototype vector of each class, classify according to the magnitude of the cosine distance, and obtain the class of each test sample, thereby realizing the monitoring of bolt loosening of the bolt connection structure actually installed on the overall structure.

[0016] Furthermore, the embedding function in step 1 employs a convolutional neural network, a long short-term memory recurrent neural network, or a multilayer feedforward neural network.

[0017] Furthermore, the embedding function in step 1 employs a convolutional neural network, which contains four convolutional modules. Each convolutional module contains one convolutional layer, one batch normalization layer, one ReLU layer, and one max pooling layer.

[0018] Furthermore, in steps 2 and 3, the preprocessing of the collected sound data is as follows: the sound loudness is normalized, the silence before and after is removed, and the data length is made consistent.

[0019] Furthermore, in step 3, the specific process of performing temporal masking data augmentation is as follows:

[0020] For the preprocessed one-dimensional sound signal, a time domain interval [t0, t0+Δt] is randomly selected, and the signal value in this interval is set to 0 to obtain new data; where the maximum selectable value of Δt is 10% of the signal length, and the minimum selectable value is one-third of the maximum selectable value.

[0021] Furthermore, in step 3, the specific process of performing frequency domain masking data enhancement is as follows:

[0022] 1) Transform the filtered one-dimensional sound signal to the frequency domain using FFT;

[0023] 2) Obtain the main lobe frequency range of the signal [f min ,f max ];

[0024] 3) In the frequency range [f min ,f max [f] cen ×95%,f cen For other regions within the range of [×105%], randomly select a frequency interval [f0, f0+Δf] and assign the FFT amplitude within this frequency interval a value of 0; where f cen For the center frequency, the maximum selectable value of Δf is 15% of the main lobe bandwidth of the signal, and the minimum selectable value is one-third of the maximum selectable value;

[0025] 4) The frequency-domain masked signal is converted to the time domain using IFFT to obtain new data.

[0026] Based on the above method, the present invention also proposes a storage medium storing a computer-executable program, which, when executed, can implement the above method.

[0027] Based on the above method and storage medium, the present invention also proposes a portable detection device for monitoring bolt loosening, including a control chip, a human-machine interface, a power supply and a sound pickup mechanism.

[0028] The sound pickup mechanism is used to collect bolt tapping sound information during the bolt loosening detection process and transmit the sound information to the control chip;

[0029] The control chip can control the sound pickup mechanism to collect sound information according to the instructions received from the human-computer interaction interface; and the control chip has a preset storage medium, which can detect the sound information collected by the sound pickup mechanism according to the instructions received from the human-computer interaction interface, obtain the detection result, and display it on the human-computer interaction interface.

[0030] The power bank supplies power to the control chip, the human-machine interface, and the microphone.

[0031] Furthermore, the control chip uses an embedded Linux board.

[0032] Furthermore, the embedded Linux board connects to the power bank and the human-machine interface via a GPIO interface, and to the microphone via a USB interface.

[0033] Beneficial effects

[0034] This invention addresses the need for portable bolt loosening monitoring by proposing a bolt loosening monitoring method and portable detection device based on few-sample learning sound recognition. Firstly, addressing the practical problem that existing methods such as deep learning require large amounts of training data, and that it is difficult to obtain a large number of labeled samples when collecting sound signals by tapping actual structures, this invention proposes using an independent experimental structure similar to the engineering object being detected to construct an auxiliary data set. Data augmentation is performed based on time-domain / frequency-domain masking, and an improved prototype network is established using an interleaved training method. This network directly uses 1D sound signals as input for few-sample learning, achieving high-precision bolt loosening recognition with very few training samples, overcoming the difficulty of deep learning recognition accuracy being heavily dependent on the size of the dataset. In terms of hardware implementation, an embedded Linux board and microphone are used to collect sound signals, enabling portable, rapid, and direct detection of bolt loosening. The device is battery-powered, portable, more practical, lower in cost, and convenient for engineering applications.

[0035] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0036] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which:

[0037] Figure 1 : Schematic diagram of the method flow of the present invention

[0038] Figure 2Schematic diagram of the experimental specimen used in this invention

[0039] Figure 3 Bolt loosening detection program GUI

[0040] Figure 4 The modified Raspberry Pi (top layer is a touchscreen, middle layer is a power bank, bottom layer is the Raspberry Pi motherboard with peripheral interfaces)

[0041] Figure 5 Diagram showing the relationship between the modified Raspberry Pi hardware. Detailed Implementation

[0042] The embodiments of the present invention are described in detail below. These embodiments are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0043] This embodiment is an application example of bolt loosening detection for connecting components under the chassis of a certain type of high-speed train. Due to the complexity of acquiring sample data for high-speed train chassis connecting components and the limited number of tagged samples that can be collected, it is a typical small sample problem. Furthermore, sound signals differ from typical image data, and traditional small sample learning methods applied to image data are not suitable for sound signals. Therefore, this embodiment proposes a bolt loosening detection method based on small sample learning sound recognition, employing an improved small sample learning model suitable for sound signals and a sound signal data enhancement method based on time-domain / frequency-domain masking. The specific steps include:

[0044] Step 1: Build a few-shot learning model in the PyTorch deep learning framework. The prototype network is the most commonly used few-shot learning model, which consists of an embedding function and a metric module.

[0045] Embedding functions can employ common neural networks, such as convolutional neural networks (CNNs), long short-term memory recurrent neural networks (LSRMs), and multilayer feedforward neural networks (MFNNs). In this embodiment, a CNN is used as the embedding function to extract features from the input signal. The constructed embedding function contains four convolutional modules, each containing one convolutional layer, one batch normalization layer, one ReLU layer, and one max-pooling layer. The output of the embedding function is the signal's embedding vector. By averaging the embedding vectors of all training samples of the same category, the prototype vector for that category can be obtained. During classification, the embedding function extracts the embedding vector of the test sample, and then the distance between the test sample's embedding vector and the prototype vectors of each category is calculated. If the distance is small, the test sample is determined to belong to the same category as the prototype vector; otherwise, they are not, thus achieving classification.

[0046] In this embodiment, the measurement module uses cosine similarity as the distance function, a special choice for audio signals. Traditional prototype network models use Euclidean distance as the distance function, but since audio signals are one-dimensional, the accuracy of classification using Euclidean distance is poor. However, by using cosine similarity as the distance function, the accuracy of bolt loosening identification is improved in small sample cases, as verified in the computational examples.

[0047] Step 2: Using the bolted connection structure actually installed on the overall structure as the target domain structure, the target domain structure is struck with a small hammer (such as a claw hammer or fitter's hammer) under different bolt torque states, and the sound data generated by the striking is collected; the collected sound data is preprocessed to obtain the target domain dataset, and the target domain dataset is divided into a support set S and a test set T, where the support set S contains training samples with category labels, and the category labels refer to the bolt torque state.

[0048] In this embodiment, a torque wrench was used to adjust the torque of the bolt to 30 N·m (tight) or 0 N·m (loose), resulting in two categories.

[0049] After collecting the sound data, the data is preprocessed, including loudness-based normalization, removal of silence before and after the sound, and standardization of data length to obtain the target domain dataset. The target domain dataset is then divided into a support set S and a test set T. S and T share the same label space. Due to limitations in practical applications, the number of available labeled samples in the support set S is very small. In this embodiment, each category has 1-5 training samples. If S contains C categories, and each category contains K labeled samples, then this type of task is called a (C-way K-shot) few-shot classification task.

[0050] Step 3: Prepare an independent source domain structure identical to the target domain structure. Tap the source domain structure under different bolt torque conditions and collect the resulting sound data. Preprocess the collected sound data to obtain an auxiliary set A. Enhance the auxiliary set A using a time-domain / frequency-domain masking data augmentation method to obtain the enhanced auxiliary set A1. Specifically: during time-domain masking data augmentation, randomly set all values ​​of a certain time period of the collected sound signal to zero, using this as new data. In frequency masking, perform a Fourier transform on the collected sound signal, then select a frequency range outside the center frequency band within the entire sound signal frequency band for masking, and finally perform an inverse Fourier transform on the masked signal, using this as new data. The auxiliary set A1 typically contains numerous non-overlapping categories, each with many labeled training samples. After training the model on the enhanced auxiliary set A1, it will achieve even higher bolt loosening detection accuracy.

[0051] In this embodiment, for the undercarriage connection component structure of a certain type of high-speed train, two perforated steel plates, each 160*160mm in size and 20mm thick, are connected by four M12 bolts as an independent source area structure. Figure 2 As shown.

[0052] In this embodiment, the specific process of performing temporal masking data enhancement is as follows:

[0053] For the preprocessed one-dimensional sound signal, a time domain interval [t0, t0+Δt] is randomly selected, and the signal value in this interval is set to 0 to obtain new data; where the maximum selectable value of Δt is 10% of the signal length, and the minimum selectable value is one-third of the maximum selectable value.

[0054] In this embodiment, the specific process of frequency domain masking data enhancement is as follows:

[0055] 1) Transform the filtered one-dimensional sound signal to the frequency domain using FFT;

[0056] 2) Obtain the main lobe frequency range of the signal [f min ,f max ];

[0057] 3) In the frequency range [f min ,f max [f] cen ×95%,f cen For other regions within the range of [×105%], randomly select a frequency interval [f0, f0+Δf] and assign the FFT amplitude within this frequency interval a value of 0; where f cen For the center frequency, the maximum selectable value of Δf is 15% of the main lobe bandwidth of the signal, and the minimum selectable value is one-third of the maximum selectable value; it can be seen here that, because the energy is concentrated at the center frequency f... cen Nearby, therefore, the data within the center frequency band, i.e., [f cen ×95%,f cen [×105%], no frequency masking is performed;

[0058] 4) The frequency-domain masked signal is converted to the time domain using IFFT to obtain new data.

[0059] Through time-domain / frequency-domain masking enhancement, the auxiliary set A is expanded by 8 times to obtain the enhanced auxiliary set A1.

[0060] Step 4: Use the interlude training method for training.

[0061] The model is trained on the auxiliary set A1 for multiple epochs:

[0062] During a training cycle, multiple auxiliary support subsets A are randomly constructed from the auxiliary set A1. sAuxiliary test subset A t Auxiliary support subset A s The categories and number of samples in the auxiliary test subset A are the same as those in the support set S. t The categories and number of samples in set A are the same as those in the test set T; a set A s and A t This constitutes an episode, in which multiple episodes are constructed to train the few-shot learning model established in step 1; at the end of the last training cycle, the trained few-shot learning model is saved for bolt loosening monitoring of the target domain structure.

[0063] It can be seen that the sampled episodes simulate the few-shot classification task, and the few-shot learning model established in step 1 is trained using multiple constructed episodes. That is, the samples from the episodes are input into the few-shot learning model in step 1 for training, thus completing one "episode training". Each epoch of model training requires sampling N episodes, meaning that after M epochs of training, the model undergoes M*N episode training. When the episode training is performed on the auxiliary test subset A... t Model training ends when the classification accuracy reaches the specified index.

[0064] Step 5: Input the training samples from the support set S into the embedding function of the few-shot learning model trained in Step 4, calculate the embedding vector of each training sample, and average the embedding vectors of all samples of the same class in the support set S to obtain the prototype vector of that class; input the samples from the test set T into the embedding function of the few-shot learning model trained in Step 4, calculate the embedding vector of each test sample, calculate the cosine distance between the embedding vector of each test sample and the prototype vector of each class, classify according to the magnitude of the cosine distance, and obtain the class of each test sample, thus realizing the monitoring of bolt loosening of bolted connection structures actually installed on the overall structure.

[0065] Based on the above method, this embodiment also proposes a portable detection device for monitoring bolt loosening, such as... Figures 3-5 As shown, it includes a control chip, a human-machine interface, a power bank, and a sound pickup mechanism.

[0066] The sound pickup mechanism is used to collect bolt tapping sound information during the bolt loosening detection process and transmit the sound information to the control chip;

[0067] The control chip can control the sound pickup mechanism to collect sound information according to the instructions received from the human-computer interaction interface; and the control chip has a preset storage medium, which can detect the sound information collected by the sound pickup mechanism according to the instructions received from the human-computer interaction interface, obtain the detection result, and display it on the human-computer interaction interface.

[0068] The power bank supplies power to the control chip, the human-machine interface, and the microphone.

[0069] In this embodiment, the control chip is an embedded Linux board (Raspberry Pi). The embedded Linux board is connected to a power bank and a human-machine interface via a GPIO interface, and to a microphone via a USB interface. A deep learning framework and its runtime environment are installed on the embedded Linux board (such as the Raspberry Pi). The small-sample bolt loosening identification model embodying the above method is ported from the computer to the Raspberry Pi. Simultaneously, a graphical user interface is developed to enable one-click operation for identifying loosening. The interface includes "Record" and "Identify" buttons to perform recording and loosening identification functions, respectively.

[0070] To use, power on the Raspberry Pi, connect the external microphone to the Raspberry Pi, and place the microphone near the bolt to be tested on the test piece. Run the bolt loosening detection program. Click the "Record" button on the program interface to start recording. Tap the bolt to be tested with a hammer; the recording is complete when the screen displays "Recording Complete." Click the "Recognize" button on the program interface, and the screen will display whether the bolt is loose.

[0071] To illustrate the effectiveness of the present invention, this embodiment uses the proposed improved prototype network few-shot model and data augmentation method to identify the bolt loosening of the structure, and compares the recognition accuracy of the original prototype network, the improved prototype network and other commonly used few-shot learning methods.

[0072] Five different bolt loosening conditions were set as shown in Table 1. Different few-shot learning methods used the same 4-layer convolutional feature extractor and hyperparameters, and were trained intermittently on the same auxiliary set. Then, (5-way 5-shot) and (5-way 1-shot) few-shot classification tasks were performed to identify the bolt loosening conditions in Table 1. The mean and standard deviation of the classification accuracy after 50 tests are shown in Table 2.

[0073] Table 1. Bolt loosening status

[0074]

[0075] Table 2 Test accuracy of different methods

[0076]

[0077] Among them, RelationNet, MatchingNet, MAML, and ProtoNet are commonly used few-shot classification methods. As can be seen from the table above, after augmenting the training data using temporal / frequency domain masking, the proposed improved ProtoNet achieves the highest classification accuracy among all models.

[0078] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention without departing from the principles and spirit of the present invention.

Claims

1. A bolt loosening monitoring method based on few-sample learning sound recognition, characterized in that: Includes the following steps: Step 1: Build a few-shot learning model; the few-shot learning model includes an embedding function and a metric module; the embedding function is implemented using a deep learning neural network, and the metric module is implemented using a cosine distance function; Step 2: Using the bolted connection structure actually installed on the overall structure as the target domain structure, the target domain structure is struck under different bolt torque states, and the sound data generated by the striking is collected; the collected sound data is preprocessed to obtain the target domain dataset, and the target domain dataset is divided into a support set S and a test set T, wherein the support set S contains training samples with category labels, and the category labels refer to the bolt torque state; Step 3: Prepare an independent source domain structure identical to the target domain structure, and tap the source domain structure under different bolt torque conditions to collect the sound data generated by the tapping; preprocess the collected sound data to obtain auxiliary set A; The auxiliary set A is enhanced using a data augmentation method based on time-domain and frequency-domain masking to obtain the enhanced auxiliary set A1. Specifically: during time-domain masking data augmentation, the values ​​of the collected sound signal for a certain time period are randomly set to zero and used as new data; in frequency masking, the collected sound signal is subjected to Fourier transform, and then a frequency interval outside the center frequency band is selected in the entire frequency band of the sound signal for masking. The masked signal is then subjected to inverse Fourier transform and used as new data. Step 4: Train the model on the auxiliary set A1 for multiple epochs; During a training cycle, multiple auxiliary support subsets A are randomly constructed from the auxiliary set A1. s and auxiliary test subset A t Auxiliary support subset A s The categories and number of samples in the auxiliary test subset A are the same as those in the support set S. t The categories and number of samples in set A are the same as those in the test set T; a set A s and A t This constitutes an episode, and the few-shot learning model built in step 1 is trained sequentially using multiple constructed episodes; A is reconstructed randomly in each training cycle. s and A t Then, the model trained in the previous cycle is used for further training; at the end of the last training cycle, the trained small sample learning model is saved for bolt loosening monitoring of the target domain structure. Step 5: Input the training samples from the support set S into the embedding function of the few-shot learning model trained in Step 4, calculate the embedding vector of each training sample, and average the embedding vectors of all samples of the same class in the support set S to obtain the prototype vector of that class; input the samples from the test set T into the embedding function of the few-shot learning model trained in Step 4, calculate the embedding vector of each test sample, calculate the cosine distance between the embedding vector of each test sample and the prototype vector of each class, classify according to the magnitude of the cosine distance, and obtain the class of each test sample, thereby realizing the monitoring of bolt loosening of the bolt connection structure actually installed on the overall structure.

2. The bolt loosening monitoring method based on few-sample learning sound recognition according to claim 1, characterized in that: The embedding function in step 1 uses a convolutional neural network, a long short-term memory recurrent neural network, or a multilayer feedforward neural network.

3. The bolt loosening monitoring method based on few-sample learning sound recognition according to claim 2, characterized in that: The embedding function in step 1 uses a convolutional neural network, which contains four convolutional modules. Each convolutional module contains one convolutional layer, one batch normalization layer, one ReLU layer, and one max pooling layer.

4. The bolt loosening monitoring method based on few-sample learning sound recognition according to claim 1, characterized in that: In steps 2 and 3, the preprocessing of the collected sound data is as follows: the sound loudness is normalized, silences before and after the sound are removed, and the data length is made consistent.

5. The bolt loosening monitoring method based on few-sample learning sound recognition according to claim 1, characterized in that: In step 3, the specific process of performing temporal masking data augmentation is as follows: For the preprocessed one-dimensional sound signal, a time domain interval [t0, t0+Δt] is randomly selected, and the signal value in this interval is set to 0 to obtain new data; where the maximum selectable value of Δt is 10% of the signal length, and the minimum selectable value is one-third of the maximum selectable value.

6. The bolt loosening monitoring method based on few-sample learning sound recognition according to claim 1, characterized in that: In step 3, the specific process of performing frequency domain masking data enhancement is as follows: 1) Transform the filtered one-dimensional sound signal to the frequency domain using FFT; 2) Obtain the main lobe frequency range of the signal [f min ,f max ]; 3) In the frequency range [f min ,f max [f] cen ×95%,f cen For other regions within the range of [×105%], randomly select a frequency interval [f0, f0+Δf] and assign the FFT amplitude within this frequency interval a value of 0; where f cen For the center frequency, the maximum selectable value of Δf is 15% of the main lobe bandwidth of the signal, and the minimum selectable value is one-third of the maximum selectable value; 4) The frequency-domain masked signal is converted to the time domain using IFFT to obtain new data.

7. A storage medium storing a computer-executable program, characterized in that: When the computer executable program is executed, it can implement the method described in any one of claims 1-6.

8. A portable detection device for monitoring bolt loosening, characterized in that: This includes control chips, human-machine interfaces, power banks, and sound pickup mechanisms; The sound pickup mechanism is used to collect bolt tapping sound information during the bolt loosening detection process and transmit the sound information to the control chip; The control chip can control the sound pickup mechanism to collect sound information according to the instructions received from the human-computer interaction interface; and the control chip is pre-loaded with the storage medium described in claim 7, which can detect the sound information collected by the sound pickup mechanism according to the instructions received from the human-computer interaction interface, obtain the detection result, and display it on the human-computer interaction interface. The power bank supplies power to the control chip, the human-machine interface, and the microphone.

9. A portable detection device for monitoring bolt loosening according to claim 8, characterized in that: The control chip uses an embedded Linux board.

10. A portable detection device for monitoring bolt loosening according to claim 9, characterized in that: The embedded Linux board connects to the power bank and human-machine interface via GPIO interface, and to the pickup mechanism via USB interface.

Images