Iterative method of bolt damage detection model and bolt damage detection method

Abstract

This invention provides an iterative method for a bolt damage detection model and a bolt damage detection method. The iterative method includes: extracting audio signals from target audio data to obtain an audio waveform diagram, and processing the audio waveform diagram to obtain an audio feature map; inputting the audio feature map into an initial damage detection model to obtain the predicted damage detection result of the target bolt output by the initial damage detection model; iterating the parameters of the initial damage detection model based on the target damage detection result and the predicted damage detection result to obtain a bolt damage detection model. The target audio data corresponding to the acoustic vibration signal obtained by striking the bolt serves as the state feedback of the bolt damage, and is used as the model input to train a bolt damage detection model for bolt damage detection. This provides a basis for rapid and accurate bolt damage detection, while ensuring the safety of the detection process, reducing detection costs, and improving detection efficiency.

Description

Technical Field

[0001] This invention relates to the field of damage detection technology, and in particular to an iterative method for bolt damage detection model and a bolt damage detection method. Background Technology

[0002] Wind energy is one of the most important clean energy sources. Currently, the connections of wind turbines in wind power plants are mostly completed using a large number of bolts. Among them, fastening bolts are important connecting components of wind turbines. Under complex stress and vibration conditions during long-term operation, bolts are prone to creep, fatigue cracks, and other problems. In addition, excessive preload during installation can also cause micro-cracks in the bolts. Furthermore, as the operating time increases, the cracks inside the bolts continue to expand, eventually leading to bolt failure. Bolt fracture and failure can cause the wind turbine tower to collapse, resulting not only in significant economic losses but also the potential for casualties.

[0003] Currently, bolt damage detection in wind turbines is mostly done manually, requiring each bolt in the turbine to be tested individually. This method is not only extremely resource-intensive but also poses significant safety hazards. Furthermore, since most of the bolts to be tested are installed on the complex structure of wind turbines, transporting damage detection equipment to high-altitude environments for testing is not feasible. Therefore, developing a method or equipment that can quickly and accurately detect bolt damage defects in wind turbine units while ensuring safety has become an urgent technical problem to be solved. Summary of the Invention

[0004] This invention provides an iterative method for bolt damage detection model and a bolt damage detection method to address the shortcomings of existing technologies that rely on manual bolt damage detection, which is not only time-consuming and labor-intensive but also poses significant safety hazards. By training an initial damage detection model using training data, a bolt damage detection model that can be used for bolt damage detection is obtained, providing a basis for rapid and accurate bolt damage detection while ensuring the safety of the detection process.

[0005] This invention provides an iterative method for a bolt damage detection model, comprising:

[0006] Acquire the target audio data and target damage detection results corresponding to the target bolt;

[0007] The target audio data is subjected to audio signal extraction to obtain an audio waveform diagram, and the audio waveform diagram is subjected to data processing to obtain an audio feature map;

[0008] The audio feature map is input into the initial damage detection model to obtain the predicted damage detection result of the target bolt output by the initial damage detection model;

[0009] Based on the target damage detection results and the predicted damage detection results, the parameters of the initial damage detection model are iterated to obtain the bolt damage detection model.

[0010] According to an iterative method for a bolt damage detection model provided by the present invention, the initial damage detection model is used to determine the audio feature vector corresponding to the audio feature map, and to split the audio feature vector. Based on the correlation between the split sub-audio feature vectors and the position feature vectors of the sub-audio feature vectors, the sub-audio feature vectors are fused. Damage detection is performed based on the fused bolt feature vector to obtain the predicted damage detection result of the target bolt.

[0011] According to an iterative method for a bolt damage detection model provided by the present invention, the step of inputting the audio feature map into an initial damage detection model to obtain the predicted damage detection result of the target bolt output by the initial damage detection model includes:

[0012] The audio feature map is input into the audio feature extraction layer in the initial damage detection model to obtain the sub-audio feature vectors output by the audio feature extraction layer;

[0013] Each of the sub-audio feature vectors is input into the audio feature fusion layer in the initial damage detection model to obtain the bolt feature vector output by the audio feature fusion layer;

[0014] The bolt feature vector is input into the damage detection layer in the initial damage detection model to obtain the predicted damage detection result output by the damage detection layer.

[0015] According to an iterative method for a bolt damage detection model provided by the present invention, the step of inputting the sub-audio feature vectors into the audio feature fusion layer in the initial damage detection model to obtain the bolt feature vector output by the audio feature fusion layer includes:

[0016] Each sub-audio feature vector is input into the position coding layer in the audio feature fusion layer to obtain the position feature vector of each sub-audio feature vector in the audio feature vector output by the position coding layer;

[0017] The sub-audio feature vectors and their position feature vectors are input into the self-attention layer in the audio feature fusion layer to obtain the bolt feature vector output by the self-attention layer.

[0018] According to the iterative method of the bolt damage detection model provided by the present invention, the audio feature extraction layer is constructed on the basis of a residual convolutional network;

[0019] The audio feature fusion layer is built on top of a self-attention network;

[0020] The damage detection layer is constructed based on a Gaussian capsule network.

[0021] The Gaussian capsule network is a capsule network with Gaussian functions as network nodes.

[0022] According to an iterative method for a bolt damage detection model provided by the present invention, the target audio data is determined based on the acoustic vibration signal generated by striking the target bolt;

[0023] The step of processing the audio waveform to obtain an audio feature map includes:

[0024] The audio waveform is cropped to obtain a first waveform that includes the acoustic vibration signal generated at the moment of impact.

[0025] The first waveform is processed to obtain bolt audio data, and the bolt audio data is then transformed into an audio feature map.

[0026] The bolt audio data includes at least one of time-domain signal, spectrogram, Mel spectrogram, and Mel cepstral coefficients.

[0027] According to the iterative method of the bolt damage detection model provided by the present invention, the parameter iteration process of the initial damage detection model uses the mean square error as the loss function, and updates and optimizes the weights and biases of each neuron in the model through the backpropagation method to obtain the bolt damage detection model.

[0028] The present invention also provides a bolt damage detection method, comprising:

[0029] An audio feature map corresponding to the bolt to be tested is determined. The audio feature map is determined based on the audio waveform map corresponding to the bolt to be tested. The audio waveform map is obtained by extracting audio signals from the audio data corresponding to the bolt to be tested.

[0030] The audio feature map is input into the bolt damage detection model to obtain the damage detection result of the bolt to be detected output by the bolt damage detection model;

[0031] The bolt damage detection model is determined based on the iterative method of the bolt damage detection model as described in any of the above.

[0032] The present invention also provides an iterative apparatus for a bolt damage detection model, comprising:

[0033] The acquisition unit is used to acquire the target audio data and target damage detection results corresponding to the target bolt.

[0034] The processing unit is used to extract audio signals from the target audio data to obtain an audio waveform diagram, and to process the audio waveform diagram to obtain an audio feature map.

[0035] The detection unit is used to input the audio feature map into the initial damage detection model to obtain the predicted damage detection result of the target bolt output by the initial damage detection model;

[0036] An iterative unit is used to iterate the parameters of the initial damage detection model based on the target damage detection result and the predicted damage detection result to obtain a bolt damage detection model.

[0037] The present invention also provides a bolt damage detection device, comprising:

[0038] The feature map determination unit is used to determine the audio feature map corresponding to the bolt to be detected. The audio feature map is determined based on the audio waveform map corresponding to the bolt to be detected. The audio waveform map is obtained by extracting audio signals based on the audio data corresponding to the bolt to be detected.

[0039] The damage detection unit is used to input the audio feature map into the bolt damage detection model to obtain the damage detection result of the bolt to be detected output by the bolt damage detection model; the bolt damage detection model is determined based on the iterative method of the bolt damage detection model as described in any of the above.

[0040] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement an iterative method for a bolt damage detection model as described above, or a bolt damage detection method as described above.

[0041] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements an iterative method for a bolt damage detection model as described above, or a bolt damage detection method as described above.

[0042] The iterative method and bolt damage detection method for the bolt damage detection model provided by this invention extract audio signals from target audio data to obtain an audio waveform diagram, and process the audio waveform diagram to obtain an audio feature map. The audio feature map is then input into an initial damage detection model to obtain the predicted damage detection result output by the initial damage detection model. Based on the target damage detection result and the predicted damage detection result, the parameters of the initial damage detection model are iterated to obtain a bolt damage detection model. The target audio data corresponding to the acoustic vibration signal obtained by striking the bolt serves as the state feedback of bolt damage, and is used as the model input to train a bolt damage detection model for bolt damage detection. This provides a basis for rapid and accurate bolt damage detection, while ensuring the safety of the detection process, reducing detection costs, and improving detection efficiency. Attached Figure Description

[0043] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0044] Figure 1 This is a flowchart illustrating the iterative method for the bolt damage detection model provided by the present invention;

[0045] Figure 2 This is a schematic diagram of the structure of the initial damage detection model provided by the present invention;

[0046] Figure 3 This is a schematic flowchart of the bolt damage detection method provided by the present invention;

[0047] Figure 4 This is a schematic diagram of the structure of the iterative device for the bolt damage detection model provided by the present invention;

[0048] Figure 5 This is a schematic diagram of the bolt damage detection device provided by the present invention;

[0049] Figure 6 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0051] This invention provides an iterative method for a bolt damage detection model. The method aims to use the target audio data corresponding to the acoustic vibration signal obtained by striking the bolt as the state feedback of the bolt damage, and use this as the model input to train a bolt damage detection model for bolt damage detection. This provides a basis for fast and accurate bolt damage detection, while ensuring the safety of the detection process, reducing detection costs, and improving detection efficiency. Figure 1 This is a flowchart illustrating the iterative method for the bolt damage detection model provided by the present invention, as shown below. Figure 1 As shown, the method includes:

[0052] Step 110: Obtain the target audio data and target damage detection results corresponding to the target bolt;

[0053] Here, the target bolt can be a fastener used for component connection in a wind turbine, such as a high-strength bolt in the wind turbine or a fastening bolt in the wind turbine itself. Once the target bolt is identified, the corresponding audio data, or target audio data, can be obtained. This target audio data can be obtained by tapping the target bolt; it is determined based on the acoustic vibration signal generated by the target bolt when tapped. Therefore, the target audio data can also be called the acoustic vibration audio data of the target bolt.

[0054] While determining the target audio data corresponding to the target bolt, it is also necessary to determine the damage condition of the target bolt and label the damage detection results, i.e., the target damage detection results. Here, the target audio data corresponding to the target bolt can be regarded as training samples, and the corresponding target damage detection results can be regarded as sample labels for the training samples. Based on these two, a bolt damage detection model for bolt damage detection can be trained.

[0055] Furthermore, it's worth noting that the target bolts, their corresponding audio data, and the target damage detection results are not singular quantities, but rather a large number sufficient to constitute a training dataset for model training. To ensure effective model training, the training samples are typically very large. In this case, for each target bolt, its corresponding audio data and target damage detection results need to be obtained. Finally, by combining the audio data and damage detection results for each target bolt, a complete training dataset can be constructed for subsequent model training to obtain the bolt damage detection model.

[0056] Step 120: Extract audio signals from the target audio data to obtain an audio waveform, and process the audio waveform to obtain an audio feature map;

[0057] Specifically, after obtaining the target audio data and target damage detection results corresponding to the target bolt in step 110, step 120 can be executed to process the training samples used as model input to obtain audio feature maps. The specific process includes:

[0058] First, the acquired target audio data can be processed to extract the audio signal and form an audio waveform. That is, the audio signal of the target audio data corresponding to the target bolt can be extracted to obtain the audio waveform corresponding to the target audio data. Specifically, the target audio data can be read here. For example, the target audio data can be read through torchaudio to obtain the audio signal and thus obtain the audio waveform corresponding to the input target audio data.

[0059] It is worth noting that in this embodiment of the invention, the format of the target audio data is not specifically limited. That is, the obtained target audio data can be any audio format, such as WAV format, MP3 (Moving Picture Experts Group Audio Layer III) format, or other formats.

[0060] Subsequently, the audio waveform can be processed to obtain a feature map in a vector form acceptable to the model, i.e., an audio feature map. In other words, the audio waveform can be transformed into an audio feature map acceptable to the model. Specifically, after obtaining the audio waveform, it can be processed to obtain various forms of audio data, such as time-domain signals, spectrograms, Mel-spectrograms, Mel-frequency cipstal coefficients (MFCCs), etc. Then, the above audio data can be transformed to obtain the audio feature map.

[0061] Step 130: Input the audio feature map into the initial damage detection model to obtain the predicted damage detection result of the target bolt output by the initial damage detection model;

[0062] Specifically, after obtaining the audio feature map through the above process, step 130 can be executed. Based on the audio feature map, damage detection is performed using a pre-built initial damage detection model to obtain the predicted damage detection result output by the model. The specific process includes:

[0063] The target audio data, determined based on the acoustic vibration signals generated by striking the target bolt, can initially reflect the damage state of the bolt. Furthermore, the resulting audio feature map, obtained through effective audio signal extraction and data processing, contains more effective information about the bolt's damage than the target audio data itself. This more information-rich audio feature map better highlights the damage condition. Moreover, the audio feature map containing this information can distinguish between different levels of damage and tightness.

[0064] Therefore, after obtaining the audio feature map corresponding to the target bolt, the pre-built initial damage detection model can be used as a reference to perform damage detection on the target bolt, thereby obtaining the predicted damage detection result of the target bolt. That is, the audio feature map can be input into the initial damage detection model, and the initial damage detection model can predict the damage of the target bolt based on the input audio feature map, thereby obtaining the predicted damage detection result output by the model.

[0065] In the initial damage detection model, when performing damage detection based on the audio feature map, the feature vector corresponding to the input audio feature map can be extracted, and then self-attention calculation can be performed on the feature vector. Finally, damage detection and confidence judgment can be performed based on the feature vector output by the self-attention mechanism, thereby obtaining the predicted damage detection result of the target bolt.

[0066] It is understandable that the output of the initial damage detection model here is essentially a probability distribution of the damage level of the target bolt, which includes multiple damage levels / damage conditions predicted by the model for the target bolt, as well as the probability / confidence level corresponding to each damage level. The predicted damage detection result of the target bolt can be determined based on each damage level and its corresponding confidence level. In this embodiment of the invention, the magnitude of the vector in the final output result is used as the confidence level, and the predicted vector with the longest magnitude is determined as the predicted damage detection result.

[0067] Furthermore, it is worth noting that before inputting the target audio data into the initial damage detection model for damage detection, it is necessary to initialize the weights and biases of the neurons in each network of the initial damage detection model. This can also be understood as randomly initializing the initial values ​​of the weights and biases of the neurons in each network of the initial damage detection model.

[0068] Step 140: Based on the target damage detection results and the predicted damage detection results, perform parameter iteration on the initial damage detection model to obtain the bolt damage detection model.

[0069] Specifically, after obtaining the predicted damage detection result of the target bolt in step 130, step 140 can be executed. Based on this, and combined with the target damage detection result of the target bolt, the initial damage detection model is trained to obtain the bolt damage detection model, which specifically includes:

[0070] First, based on the target damage detection results and the predicted damage detection results, the loss of the initial damage detection model when performing damage detection is determined. This process is essentially based on the sample labels of the training samples and the prediction results to measure the model loss of the initial damage detection model. Specifically, here we can judge the distance between the target damage detection results and the predicted damage results output by the model, that is, the closeness or difference between the damage status / damage degree of the target bolt represented by the two. The larger the distance, the greater the gap between the prediction results and the sample labels, and the greater the model damage. Conversely, the smaller the distance, the smaller the gap between the prediction results and the sample labels, and the smaller the model damage.

[0071] Subsequently, based on this model loss, the parameters of the initial damage detection model can be iterated to obtain the bolt damage detection model. That is, the model parameters of the initial damage detection model can be adjusted using this model loss so that the predicted damage detection result output by the adjusted model can be as close as possible to the target damage detection result, and finally the trained bolt damage detection model is obtained.

[0072] Specifically, this model loss can be used as a benchmark to update and optimize the neuron weights and biases of each network in the initial damage detection model through backpropagation, so as to reduce the difference between the model output and the sample label. After multiple rounds of updates and iterations, the prediction results of the updated model output can be very close to or even completely consistent with the sample label. At this time, the model output meets the preset convergence condition. The model converges, and the weights and biases of the neurons in each network in the model can be saved to complete the training of the model.

[0073] The iterative method for bolt damage detection model provided by this invention extracts audio signals from target audio data to obtain an audio waveform, and processes the audio waveform to obtain an audio feature map. The audio feature map is then input into an initial damage detection model to obtain the predicted damage detection result output by the initial damage detection model. Based on the target damage detection result and the predicted damage detection result, the initial damage detection model is iterated to obtain a bolt damage detection model. The target audio data corresponding to the acoustic vibration signal obtained by striking the bolt serves as the state feedback of bolt damage, and is used as the model input to train a bolt damage detection model for bolt damage detection. This provides a basis for rapid and accurate bolt damage detection, while ensuring the safety of the detection process, reducing detection costs, and improving detection efficiency.

[0074] Based on the above embodiments, the initial damage detection model is used to determine the audio feature vector corresponding to the audio feature map, and to split the audio feature vector. Based on the correlation between the split sub-audio feature vectors and the position feature vectors of each sub-audio feature vector, the sub-audio feature vectors are fused. Damage detection is performed based on the fused bolt feature vector to obtain the predicted damage detection result of the target bolt.

[0075] Specifically, the process described above, which involves applying an initial damage model to perform damage detection on the target bolt based on the audio feature map corresponding to the target bolt, in order to obtain the predicted damage detection result of the target bolt, may include:

[0076] Understandably, after inputting the audio feature map into the initial damage detection model, the initial damage detection model first needs to determine the audio feature vector corresponding to the input audio feature map. That is, by performing feature extraction on the input data, the audio information about the bolt damage of the target bolt contained in the audio feature map can be extracted, thereby obtaining the audio feature vector corresponding to the target bolt.

[0077] Subsequently, the initial damage detection model can perform self-attention calculation on this audio feature vector, thereby obtaining the feature vector output by the self-attention mechanism. That is, we can first determine each sub-vector corresponding to the audio feature vector, that is, the sub-audio feature vectors obtained by splitting the audio feature vector. Then, we can perform self-attention calculation on each sub-audio feature vector. Here, the essence of self-attention calculation is to calculate the similarity between each sub-audio feature vector and other sub-audio feature vectors, and then perform weighted summation on each sub-audio feature vector based on these similarities, so as to obtain the bolt feature vector that integrates information from different positions.

[0078] Specifically, the initial damage detection model first determines the positional feature vectors corresponding to each sub-audio feature vector through positional encoding. That is, positional encoding extracts the positional features corresponding to each sub-audio feature vector to the audio feature vector, thereby obtaining the positional feature vectors corresponding to each sub-audio feature vector. Then, based on the positional feature vectors of each sub-audio feature vector, the sub-audio feature vectors are fused according to the correlation between them to obtain the bolt feature vector. In other words, based on the correlation between the sub-audio feature vectors obtained by splitting the audio feature vectors, the positional feature vectors corresponding to each sub-audio feature vector are fused to obtain the bolt feature vector of the target bolt.

[0079] Then, the initial damage detection model can perform damage detection on the target bolt based on the bolt feature vector to obtain the predicted damage detection result of the target bolt. That is, damage detection and confidence judgment can be performed based on the bolt feature vector of the target bolt to obtain the damage degree probability distribution output by the model. Based on the damage degree probability distribution, the predicted damage detection result of the target bolt can be obtained.

[0080] Based on the above embodiments, the audio feature map is input into the initial damage detection model to obtain the predicted damage detection result of the target bolt output by the initial damage detection model, including:

[0081] The audio feature map is input into the audio feature extraction layer in the initial damage detection model to obtain the sub-audio feature vectors output by the audio feature extraction layer;

[0082] Each sub-audio feature vector is input into the audio feature fusion layer in the initial damage detection model to obtain the bolt feature vector output by the audio feature fusion layer;

[0083] The bolt feature vector is input into the damage detection layer in the initial damage detection model to obtain the predicted damage detection result output by the damage detection layer.

[0084] Specifically, step 130, which involves inputting the audio feature map into the initial damage detection model to obtain the predicted damage detection result of the target bolt output by the initial damage detection model, may include the following steps:

[0085] Figure 2 This is a schematic diagram of the structure of the initial damage detection model provided by the present invention, as shown below. Figure 2 As shown, the initial damage detection model includes an audio feature extraction layer, an audio feature fusion layer, and a damage detection layer. The audio feature extraction layer extracts features from the input audio feature map and can split the extracted audio feature vectors into sub-audio feature vectors. The audio feature fusion layer fuses the sub-audio feature vectors based on the correlation between them and their positional features, thus obtaining the bolt feature vector. The damage detection layer performs damage detection on the bolt feature vector of the target bolt, obtaining the predicted damage detection result for the target bolt.

[0086] Specifically, when performing damage detection using the initial damage detection model, the audio feature map corresponding to the target bolt can first be input into the audio feature extraction layer of the initial damage detection model. The audio feature extraction layer then performs feature extraction, thereby obtaining the audio feature vector output by the audio feature extraction layer. That is, the audio feature extraction layer can extract features from the audio feature map input to this layer to extract the audio information about bolt damage contained in the audio feature map, thereby obtaining the audio feature vector corresponding to the target bolt. Afterward, this audio feature vector can be split into multiple sub-vectors to obtain each sub-audio feature vector corresponding to the audio feature vector.

[0087] Subsequently, the sub-audio feature vectors output by the audio feature extraction layer can be input into the audio feature fusion layer in the initial damage detection model. This layer contains two computational parts: position feature vector calculation and self-attention calculation. The audio feature fusion layer first calculates the position feature vectors of each sub-audio feature vector, that is, it encodes the position of each sub-audio feature vector to determine the position feature vector corresponding to each sub-audio feature vector through position encoding. Then, it performs self-attention calculation, that is, based on the position feature vectors of each sub-audio feature vector, it fuses the sub-audio feature vectors according to the correlation between each sub-audio feature vector, thereby obtaining the bolt feature vector of the target bolt.

[0088] Then, the bolt feature vector output by the audio feature fusion layer can be input into the damage detection layer in the initial damage detection model. The damage detection layer can perform damage detection on the target bolt based on the input bolt feature vector, thereby obtaining the predicted damage detection result of the target bolt. That is, damage detection and confidence judgment can be performed based on the bolt feature vector of the target bolt, thereby obtaining the damage degree probability distribution output by the model. Based on the damage degree probability distribution, the predicted damage detection result of the target bolt can be obtained.

[0089] Based on the above embodiments, each sub-audio feature vector is input into the audio feature fusion layer in the initial damage detection model to obtain the bolt feature vector output by the audio feature fusion layer, including:

[0090] Each sub-audio feature vector is input into the position coding layer in the audio feature fusion layer to obtain the position feature vector of each sub-audio feature vector in the audio feature vector output by the position coding layer;

[0091] Each sub-audio feature vector and its position feature vector are input into the self-attention layer in the audio feature fusion layer to obtain the bolt feature vector output by the self-attention layer.

[0092] Specifically, the process of inputting the audio feature vector into the audio feature fusion layer of the initial damage detection model to obtain the bolt feature vector output by the audio feature fusion layer includes:

[0093] In this embodiment of the invention, the audio feature fusion layer in the initial damage detection model consists of two computational parts: a position feature vector calculation part and a self-attention calculation part, namely a position encoding layer and a self-attention layer. The position encoding layer is used to encode the position of each sub-audio feature vector to obtain the position feature vector corresponding to each sub-audio feature vector. The self-attention layer is used to fuse the sub-audio feature vectors based on the correlation between each sub-audio feature vector and the position feature vector corresponding to each sub-audio feature vector to obtain the bolt feature vector of the target bolt.

[0094] Therefore, when obtaining the bolt feature vector through audio feature extraction layer fusion, each sub-audio feature vector can first be input into the position encoding layer in the audio feature fusion layer. The position encoding layer performs position encoding, thereby obtaining the position feature vector corresponding to each sub-audio feature vector. That is, the position feature vector corresponding to each sub-audio feature vector can be determined by position encoding. In other words, the position feature vector of each sub-audio feature vector is calculated by performing position feature vector calculation on each sub-audio feature vector, thereby obtaining the position feature vector of each sub-audio feature vector output by the position encoding layer.

[0095] Then, the sub-audio feature vectors and the position feature vectors corresponding to the sub-audio feature vectors output by the position coding layer can be input into the self-attention layer. The self-attention layer performs self-attention calculation, and finally the bolt feature vector of the target bolt output by the self-attention layer can be obtained. That is, the self-attention layer can calculate the correlation between the sub-audio feature vectors and fuse the position feature vectors of the sub-audio feature vectors to obtain the bolt feature vector of the target bolt.

[0096] Based on the above embodiments, the audio feature extraction layer is constructed on the basis of a residual convolutional network; the audio feature fusion layer is constructed on the basis of a self-attention network; the damage detection layer is constructed on the basis of a Gaussian capsule network; the Gaussian capsule network is a capsule network with Gaussian functions as network nodes.

[0097] Specifically, the initial damage detection model includes an audio feature extraction layer, an audio feature fusion layer, and a damage detection layer, where each layer corresponds to a network in the initial damage detection model; the initial damage detection model is constructed based on a residual convolutional network, a self-attention network, and a Gaussian capsule network.

[0098] The audio feature extraction layer corresponds to a residual convolutional network, meaning it is built upon a residual convolutional network. In this embodiment, considering the degradation problem in deep neural networks—that is, the performance of the model decreases as the number of network layers increases—a residual convolutional network is introduced to construct the audio feature extraction layer. The main idea of ​​the residual convolutional network is to solve the degradation problem through residual blocks. Each residual block contains two convolutional layers and one skip connection. The skip connection directly adds the input to the output of the residual block, allowing information to be directly passed to subsequent layers of the network, effectively solving the problems of gradient vanishing and information loss. Furthermore, the residual blocks employ batch normalization and activation functions to further improve the feature extraction capability of the audio feature extraction layer, making the extracted audio feature vector more accurate in representing the damage of the target bolt.

[0099] The audio feature fusion layer corresponds to a self-attention network, meaning it's built upon it. A self-attention network is a Vision in Transformer (VIT) network that extracts visual features from the input audio feature vector. It divides the input audio feature vector into multiple sub-vectors and then encodes each sub-audio feature vector through multiple self-attention layers and feed-forward layers, thus obtaining the global contextual information of each sub-audio feature vector. Here, the self-attention calculation in the self-attention network essentially calculates the similarity (attention score) between each sub-audio feature vector and all other sub-audio feature vectors, and then performs a weighted sum based on these similarities to obtain a fused feature vector with information from different positions. VIT uses a multi-head self-attention mechanism, dividing each sub-audio feature vector into multiple sub-vectors (heads), calculating self-attention for each head separately, and then concatenating the results from different heads. This increases the model's expressive power and parallelism.

[0100] Here, VIT's self-attention mechanism can effectively capture long-range dependencies in feature vectors without using local operations such as convolution and pooling, thus offering greater flexibility and generalization. Specifically, in this embodiment of the invention, after extracting the audio feature vector through the residual convolutional network, it can be split according to the channel direction of the vector to serve as input to the self-attention network for self-attention calculation.

[0101] The damage detection layer corresponds to a Gaussian capsule network, meaning it is built upon a Gaussian capsule network. Here, the Gaussian capsule network is a capsule network with Gaussian functions as network nodes; essentially, it's a network routing algorithm based on Gaussian functions. In this embodiment, the core of the Gaussian capsule network is to treat each node in the network as a Gaussian function, which has a center point and a standard deviation. Each node can send information to other nodes; this information can be some form of data packet or control information, with the purpose of transmitting information to other nodes in the network. In the Gaussian capsule network, routing between nodes is achieved by calculating the intersection points between Gaussian functions. These intersection points are called "hotspots," representing the shortest path between nodes. When a node needs to send information to other nodes in the network, it calculates the intersection point between its own Gaussian function and the target node's Gaussian function and sends the data packet to the nearest intersection point; this process is called hotspot routing.

[0102] Here, Gaussian capsule networks can handle highly nonlinear and complex topologies in networks and can quickly adapt to changes in nodes and connections. In this embodiment of the invention, Gaussian capsule networks are introduced and used for damage detection to obtain the final output predicted damage detection result. In this process, the magnitude of the vector in the final output result is used as the confidence level, and the prediction vector with the longest magnitude is determined as the predicted damage detection result.

[0103] Based on the above embodiments, the target audio data is determined based on the acoustic vibration signal generated by striking the target bolt;

[0104] Data processing is performed on the audio waveform to obtain an audio feature map, including:

[0105] The audio waveform is cropped to obtain a first waveform that includes the acoustic vibration signal generated at the moment of impact.

[0106] The first waveform is processed to obtain bolt audio data, and the bolt audio data is then transformed into an audio feature map.

[0107] Bolt audio data includes at least one of time-domain signal, spectrogram, Mel spectrogram, and Mel cepstral coefficients.

[0108] Specifically, step 120, which involves data processing of the audio waveform to obtain the audio feature map, includes the following steps:

[0109] First, the audio waveform obtained by extracting the audio signal can be cropped to obtain a first waveform, which is a waveform containing the acoustic vibration signal generated at the moment of impact. Specifically, after obtaining the audio waveform by extracting the audio signal, the audio waveform can be cropped according to a pre-set cropping rule adapted to the acoustic vibration signal generated at the moment of impact, so as to crop it into waveform data containing the acoustic signal of vibration emitted at the moment of impact, thereby obtaining the first waveform.

[0110] Next, the first waveform can be processed to obtain bolt audio data. This bolt audio data can be any one of the following, or multiple types: time-domain signal, spectrogram, Mel spectrogram, or Mel cepstral coefficients. Therefore, after obtaining the first waveform, the bolt audio data corresponding to the target bolt can be obtained through time-domain signal extraction, frequency-domain conversion, and fast Fourier transform. Then, this bolt audio data can be converted into a feature map in a vector form acceptable to the initial damage detection model. In other words, an audio feature map acceptable to the model can be obtained by performing feature map conversion on the bolt audio data.

[0111] Based on the above embodiments, the initial damage detection model uses mean squared error as the loss function during parameter iteration. The weights and biases of each neuron in the model are updated and optimized through backpropagation to obtain the bolt damage detection model.

[0112] Specifically, in the above process, when iterating the parameters of the initial damage detection model using the target damage detection results and the predicted damage detection results, the model loss of the initial damage detection model can be measured using the target damage detection results and the predicted damage detection results. That is, the mean square error can be used as the loss function of the initial damage detection model. Based on the target damage detection results and the predicted damage detection results, the model loss can be calculated, and on this basis, the parameters of the model can be iterated through the backpropagation method to obtain the bolt damage detection model.

[0113] Specifically, the mean square error can be calculated based on the target damage detection results and the predicted damage detection results of the target bolt, thereby obtaining the model loss of the initial damage detection model. Since the loss is measured by the mean square error as the loss function, the model loss calculated here can also be called the mean square loss.

[0114] In this embodiment of the invention, Mean Squared Loss (MSE Loss) is chosen as the loss function for the model, instead of the conventional Cross Entropy Loss. In the final Gaussian capsule network, MSE Loss is used to calculate the routing weights between different capsules. Since each capsule can be considered a Gaussian function, the Euclidean distance between any capsule and the target capsule can be calculated. MSE Loss uses the squared Euclidean distance as the loss function to measure the difference between the predicted result and the sample label. That is, if the target capsule's position is t and the predicted position is p, the MSE Loss can be expressed as:

[0115] MSE Loss = ||pt|| 2

[0116] Here, |pt| represents the Euclidean distance between the predicted location and the target location. In Gaussian capsule networks, when calculating routing weights using MSE Loss, the predicted location can be considered as the routing weight between capsules, and the target location as the true weight between capsules. By optimizing MSE Loss, the routing weights between different capsules can be adjusted to obtain more accurate prediction results.

[0117] Here, the smaller the MSE Loss, the smaller the difference between the predicted result and the sample label, and the more accurate the prediction result; conversely, the larger the MSE Loss, the greater the difference between the predicted result and the sample label, and the less accurate the prediction result.

[0118] Subsequently, the parameters of the initial damage detection model can be adjusted based on this mean square loss to obtain the bolt damage detection model. That is, the weights and biases of each neuron in the initial damage detection model can be updated and optimized using the backpropagation method based on this mean square loss to reduce the difference between the model output and the sample label. After multiple rounds of updates and iterations, the prediction results of the updated model output can be very close to or even completely consistent with the sample label. At this time, the model output meets the preset convergence condition. The model converges, and the weights and biases of the neurons in each network in the model can be saved to obtain the bolt damage detection model.

[0119] The present invention also provides a method for detecting bolt damage. Figure 3 This is a schematic flowchart of the bolt damage detection method provided by the present invention, as shown below. Figure 3 As shown, the method includes:

[0120] Step 310: Determine the audio feature map corresponding to the bolt to be tested. The audio feature map is determined based on the audio waveform map corresponding to the bolt to be tested. The audio waveform map is obtained by extracting audio signals from the audio data corresponding to the bolt to be tested.

[0121] Step 320: Input the audio feature map into the bolt damage detection model to obtain the damage detection result of the bolt to be detected output by the bolt damage detection model; the bolt damage detection model is determined based on the iterative method of the bolt damage detection model as described in any of the above.

[0122] Specifically, before conducting bolt damage detection, it is necessary to first determine the object to be detected, i.e. the bolt to be detected. In practical applications, the bolts to be detected are mostly high-strength bolts in wind turbine units and fastening bolts in wind turbines.

[0123] After identifying the bolt to be tested, it is necessary to determine the corresponding audio feature map. This audio feature map is determined based on the audio data corresponding to the bolt to be tested. Specifically, audio signals are extracted from the audio data corresponding to the bolt to be tested to obtain an audio waveform. Based on the audio waveform, the corresponding audio feature map of the bolt to be tested can be determined.

[0124] Specifically, after identifying the bolt to be tested, the first step is to obtain the corresponding audio data of the bolt. This audio data can be obtained by tapping the bolt, which is determined based on the acoustic vibration signal generated by the bolt when it is tapped. Therefore, the audio data can also be referred to as the acoustic vibration audio data of the bolt to be tested.

[0125] Here, the obtained audio data can be in any audio format, such as WAV, MP3 (Moving Picture Experts Group Audio Layer III), or other formats. This embodiment of the invention does not specifically limit this.

[0126] Subsequently, the acquired audio data can be processed to extract the audio signals and form an audio waveform. In other words, the audio signals of the audio data corresponding to the bolt to be tested can be extracted to obtain the audio waveform corresponding to the audio data. Specifically, the audio data can be read here. For example, the audio data can be read through torchaudio to obtain the audio signals and thus obtain the audio waveform corresponding to the input audio data.

[0127] Then, the audio waveform can be processed to obtain a feature map in a vector form acceptable to the model, i.e., an audio feature map. Specifically, the audio waveform can be transformed into an audio feature map acceptable to the model. After obtaining the audio waveform, it can be processed to obtain various forms of bolt audio data, such as time-domain signals, spectrograms, Mel-spectrograms, Mel-frequency cipstal coefficients (MFCCs), etc. Then, the bolt audio data can be transformed into a feature map to obtain an audio feature map acceptable to the model.

[0128] Furthermore, since the audio data determined based on the acoustic vibration signal generated by striking the target bolt can initially reflect the damage state of the bolt under test, and the resulting audio feature map obtained through effective audio signal extraction and data processing contains more effective information about the damage condition of the bolt under test than the audio data itself, the audio feature map containing more effective information can better highlight the damage condition. Moreover, the audio feature map containing effective information can distinguish between different types of damage and tightness.

[0129] Therefore, after obtaining the audio feature map corresponding to the bolt to be detected, the bolt damage detection model can be applied to the pre-trained model based on this audio feature map to perform damage detection on the bolt to be detected, thereby obtaining the damage detection result of the bolt to be detected. That is, the audio feature map can be input into the bolt damage detection model, and the bolt damage detection model can predict the damage of the bolt to be detected based on the input audio feature map, thereby obtaining the damage detection result output by the model.

[0130] In the bolt damage detection model, when performing damage detection based on audio feature maps, the first step is to extract the audio feature vector corresponding to the input audio feature map and then split the audio feature vector to obtain each sub-audio feature vector. Next, based on the correlation between the sub-audio feature vectors and their positional feature vectors, the sub-audio feature vectors are fused to obtain the bolt feature vector of the bolt to be detected. Finally, based on this bolt feature vector, damage detection can be performed on the bolt to obtain the predicted damage detection result for the target bolt.

[0131] Here, the output of the bolt damage detection model is essentially the probability distribution of the damage level of the bolt to be tested. This includes the model's prediction of various damage levels / damage conditions for the bolt, as well as the probability / confidence level corresponding to each damage level. The damage detection result of the bolt can then be determined based on each damage level and its corresponding confidence level.

[0132] It is worth noting that before inputting the audio feature map into the bolt damage detection model for damage detection, the bolt damage detection model needs to be pre-trained. In this embodiment of the invention, the bolt damage detection model is trained based on the iterative method of the aforementioned bolt damage detection model, and the specific training process has been described in detail above.

[0133] The bolt damage detection method provided by this invention inputs the audio feature map corresponding to the bolt to be detected into a bolt damage detection model to obtain the damage detection result of the bolt to be detected output by the bolt damage detection model. The audio feature map is determined based on the audio waveform map corresponding to the bolt to be detected, and the audio waveform map is obtained by extracting audio signals from the audio data corresponding to the bolt to be detected. The bolt damage detection model is determined based on the iterative method of the bolt damage detection model as described above. Based on the audio data corresponding to the bolt to be detected, damage detection is performed through a pre-trained bolt damage detection model. This overcomes the shortcomings of traditional methods that rely on manual bolt damage detection, which is not only time-consuming and labor-intensive but also poses significant safety hazards. It achieves rapid and accurate damage detection while ensuring the safety of the detection process, improving the efficiency of bolt damage detection, and reducing the maintenance cost of wind turbine units.

[0134] The iterative apparatus for the bolt damage detection model provided by the present invention will be described below. The iterative apparatus for the bolt damage detection model described below can be referred to in correspondence with the iterative method for the bolt damage detection model described above.

[0135] Figure 4 This is a schematic diagram of the iterative device for the bolt damage detection model provided by the present invention, as shown below. Figure 4 As shown, the device includes:

[0136] The acquisition unit 410 is used to acquire the target audio data and target damage detection results corresponding to the target bolt;

[0137] The processing unit 420 is used to extract audio signals from the target audio data to obtain an audio waveform diagram, and to process the audio waveform diagram to obtain an audio feature map.

[0138] The detection unit 430 is used to input the audio feature map into the initial damage detection model to obtain the predicted damage detection result of the target bolt output by the initial damage detection model;

[0139] The iteration unit 440 is used to perform parameter iteration on the initial damage detection model based on the target damage detection result and the predicted damage detection result to obtain the bolt damage detection model.

[0140] The iterative device for the bolt damage detection model provided by this invention extracts audio signals from target audio data to obtain an audio waveform diagram, and processes the audio waveform diagram to obtain an audio feature map. The audio feature map is then input into an initial damage detection model to obtain a predicted damage detection result output by the initial damage detection model. Based on the target damage detection result and the predicted damage detection result, the initial damage detection model is iterated to obtain a bolt damage detection model. The target audio data corresponding to the acoustic vibration signal obtained by striking the bolt serves as the state feedback of the bolt damage, and is used as the model input to train a bolt damage detection model for bolt damage detection. This provides a basis for rapid and accurate bolt damage detection, while ensuring the safety of the detection process, reducing detection costs, and improving detection efficiency.

[0141] Based on the above embodiments, the initial damage detection model is used to determine the audio feature vector corresponding to the audio feature map, and to split the audio feature vector. Based on the correlation between the split sub-audio feature vectors and the position feature vectors of the sub-audio feature vectors, the sub-audio feature vectors are fused. Damage detection is performed based on the fused bolt feature vector to obtain the predicted damage detection result of the target bolt.

[0142] Based on the above embodiments, the detection unit 430 is used for:

[0143] The audio feature map is input into the audio feature extraction layer in the initial damage detection model to obtain the sub-audio feature vectors output by the audio feature extraction layer;

[0144] Each of the sub-audio feature vectors is input into the audio feature fusion layer in the initial damage detection model to obtain the bolt feature vector output by the audio feature fusion layer;

[0145] The bolt feature vector is input into the damage detection layer in the initial damage detection model to obtain the predicted damage detection result output by the damage detection layer.

[0146] Based on the above embodiments, the detection unit 430 is used for:

[0147] Each sub-audio feature vector is input into the position coding layer in the audio feature fusion layer to obtain the position feature vector of each sub-audio feature vector in the audio feature vector output by the position coding layer;

[0148] The sub-audio feature vectors and their position feature vectors are input into the self-attention layer in the audio feature fusion layer to obtain the bolt feature vector output by the self-attention layer.

[0149] Based on the above embodiments, the audio feature extraction layer is constructed on the basis of a residual convolutional network;

[0150] The audio feature fusion layer is built on top of a self-attention network;

[0151] The damage detection layer is constructed based on a Gaussian capsule network.

[0152] The Gaussian capsule network is a capsule network with Gaussian functions as network nodes.

[0153] Based on the above embodiments, the target audio data is determined based on the acoustic vibration signal generated by striking the target bolt; the processing unit 420 is used for:

[0154] The audio waveform is cropped to obtain a first waveform that includes the acoustic vibration signal generated at the moment of impact.

[0155] The first waveform is processed to obtain bolt audio data, and the bolt audio data is then transformed into an audio feature map.

[0156] The bolt audio data includes at least one of time-domain signal, spectrogram, Mel spectrogram, and Mel cepstral coefficients.

[0157] Based on the above embodiments, the parameter iteration process of the initial damage detection model uses the mean squared error as the loss function, and updates and optimizes the weights and biases of each neuron in the model through the backpropagation method to obtain the bolt damage detection model.

[0158] The bolt damage detection device provided by the present invention is described below. The bolt damage detection device described below can be referred to in correspondence with the bolt damage detection method described above.

[0159] Figure 5 This is a schematic diagram of the bolt damage detection device provided by the present invention, as shown below. Figure 5 As shown, the device includes:

[0160] The feature map determination unit 510 is used to determine the audio feature map corresponding to the bolt to be detected. The audio feature map is determined based on the audio waveform map corresponding to the bolt to be detected. The audio waveform map is obtained by extracting audio signals based on the audio data corresponding to the bolt to be detected.

[0161] The damage detection unit 520 is used to input the audio feature map into the bolt damage detection model to obtain the damage detection result of the bolt to be detected output by the bolt damage detection model; the bolt damage detection model is determined based on the iterative method of the bolt damage detection model as described above.

[0162] The bolt damage detection device provided by this invention inputs the audio feature map corresponding to the bolt to be detected into the bolt damage detection model to obtain the damage detection result of the bolt to be detected output by the bolt damage detection model. The audio feature map is determined based on the audio waveform map corresponding to the bolt to be detected, and the audio waveform map is obtained by extracting audio signals from the audio data corresponding to the bolt to be detected. The bolt damage detection model is determined based on the iterative method of the bolt damage detection model as described above. Based on the audio data corresponding to the bolt to be detected, damage detection is performed through a pre-trained bolt damage detection model. This overcomes the shortcomings of traditional methods that rely on manual bolt damage detection, which is not only time-consuming and labor-intensive but also poses significant safety hazards. It achieves rapid and accurate damage detection while ensuring the safety of the detection process, improving the efficiency of bolt damage detection, and reducing the maintenance cost of wind turbine units.

[0163] Figure 6 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 6As shown, the electronic device may include a processor 610, a communication interface 620, a memory 630, and a communication bus 640. The processor 610, communication interface 620, and memory 630 communicate with each other via the communication bus 640. The processor 610 can call logical instructions in the memory 630 to execute an iterative method for a bolt damage detection model or a bolt damage detection method. The iterative method for the bolt damage detection model includes: acquiring target audio data and target damage detection results corresponding to the target bolt; extracting audio signals from the target audio data to obtain an audio waveform, and processing the audio waveform to obtain an audio feature map; inputting the audio feature map into an initial damage detection model to obtain a predicted damage detection result for the target bolt output by the initial damage detection model; and iterating the parameters of the initial damage detection model based on the target damage detection result and the predicted damage detection result to obtain the bolt damage detection model. The bolt damage detection method includes: determining an audio feature map corresponding to the bolt to be detected, wherein the audio feature map is determined based on an audio waveform map corresponding to the bolt to be detected, and the audio waveform map is obtained by extracting audio signals from the audio data corresponding to the bolt to be detected; inputting the audio feature map into a bolt damage detection model to obtain the damage detection result of the bolt to be detected output by the bolt damage detection model; wherein the bolt damage detection model is determined based on an iterative method of the bolt damage detection model as described above.

[0164] Furthermore, the logical instructions in the aforementioned memory 630 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0165] On the other hand, the present invention also provides a computer program product, the computer program product comprising a computer program stored on a non-transitory computer-readable storage medium, the computer program comprising program instructions, wherein when the program instructions are executed by a computer, the computer is able to execute the iterative method or bolt damage detection method provided by the above methods for executing a bolt damage detection model, wherein the iterative method for the bolt damage detection model comprises: acquiring target audio data and target damage detection results corresponding to a target bolt; extracting audio signals from the target audio data to obtain an audio waveform diagram, and performing data processing on the audio waveform diagram to obtain an audio feature diagram; inputting the audio feature diagram into an initial damage detection model to obtain a predicted damage detection result of the target bolt output by the initial damage detection model; and performing parameter iteration on the initial damage detection model based on the target damage detection result and the predicted damage detection result to obtain a bolt damage detection model. The bolt damage detection method includes: determining an audio feature map corresponding to the bolt to be detected, wherein the audio feature map is determined based on an audio waveform map corresponding to the bolt to be detected, and the audio waveform map is obtained by extracting audio signals from the audio data corresponding to the bolt to be detected; inputting the audio feature map into a bolt damage detection model to obtain the damage detection result of the bolt to be detected output by the bolt damage detection model; wherein the bolt damage detection model is determined based on an iterative method of the bolt damage detection model as described above.

[0166] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements an iterative method or bolt damage detection method for executing the bolt damage detection model provided by the methods described above. The iterative method for the bolt damage detection model includes: acquiring target audio data and target damage detection results corresponding to the target bolt; extracting audio signals from the target audio data to obtain an audio waveform diagram, and performing data processing on the audio waveform diagram to obtain an audio feature map; inputting the audio feature map into an initial damage detection model to obtain a predicted damage detection result of the target bolt output by the initial damage detection model; and performing parameter iteration on the initial damage detection model based on the target damage detection result and the predicted damage detection result to obtain a bolt damage detection model. The bolt damage detection method includes: determining an audio feature map corresponding to the bolt to be detected, wherein the audio feature map is determined based on an audio waveform map corresponding to the bolt to be detected, and the audio waveform map is obtained by extracting audio signals from the audio data corresponding to the bolt to be detected; inputting the audio feature map into a bolt damage detection model to obtain the damage detection result of the bolt to be detected output by the bolt damage detection model; wherein the bolt damage detection model is determined based on an iterative method of the bolt damage detection model as described above.

[0167] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0168] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0169] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An iterative method for a bolt damage detection model, characterized in that, include: Acquire the target audio data and target damage detection results corresponding to the target bolt; The target audio data is subjected to audio signal extraction to obtain an audio waveform diagram, and the audio waveform diagram is subjected to data processing to obtain an audio feature map; The audio feature map is input into the initial damage detection model to obtain the predicted damage detection result of the target bolt output by the initial damage detection model; Based on the target damage detection results and the predicted damage detection results, the parameters of the initial damage detection model are iterated to obtain the bolt damage detection model; The initial damage detection model is used to determine the audio feature vector corresponding to the audio feature map, and to split the audio feature vector. Based on the correlation between the split sub-audio feature vectors and the position feature vectors of the sub-audio feature vectors, the sub-audio feature vectors are fused. Damage detection is performed based on the fused bolt feature vector to obtain the predicted damage detection result of the target bolt. The step of inputting the audio feature map into the initial damage detection model to obtain the predicted damage detection result of the target bolt output by the initial damage detection model includes: The audio feature map is input into the audio feature extraction layer in the initial damage detection model to obtain the sub-audio feature vectors output by the audio feature extraction layer; Each of the sub-audio feature vectors is input into the audio feature fusion layer in the initial damage detection model to obtain the bolt feature vector output by the audio feature fusion layer; The bolt feature vector is input into the damage detection layer in the initial damage detection model to obtain the predicted damage detection result output by the damage detection layer. The audio feature extraction layer is built on top of a residual convolutional network; The audio feature fusion layer is built on top of a self-attention network; The damage detection layer is constructed based on a Gaussian capsule network. The Gaussian capsule network is a capsule network with Gaussian functions as network nodes.

2. The iterative method for the bolt damage detection model according to claim 1, characterized in that, The step of inputting each sub-audio feature vector into the audio feature fusion layer in the initial damage detection model to obtain the bolt feature vector output by the audio feature fusion layer includes: Each sub-audio feature vector is input into the position coding layer in the audio feature fusion layer to obtain the position feature vector of each sub-audio feature vector in the audio feature vector output by the position coding layer; The sub-audio feature vectors and their position feature vectors are input into the self-attention layer in the audio feature fusion layer to obtain the bolt feature vector output by the self-attention layer.

3. The iterative method for the bolt damage detection model according to claim 1 or 2, characterized in that, The target audio data is determined based on the acoustic vibration signal generated by striking the target bolt; The step of processing the audio waveform to obtain an audio feature map includes: The audio waveform is cropped to obtain a first waveform that includes the acoustic vibration signal generated at the moment of impact. The first waveform is processed to obtain bolt audio data, and the bolt audio data is then transformed into an audio feature map. The bolt audio data includes at least one of time-domain signal, spectrogram, Mel spectrogram, and Mel cepstral coefficients.

4. The iterative method for the bolt damage detection model according to claim 1 or 2, characterized in that, The initial damage detection model uses mean squared error as the loss function during parameter iteration. The weights and biases of each neuron in the model are updated and optimized through backpropagation to obtain the bolt damage detection model.

5. A method for detecting bolt damage, characterized in that, include: An audio feature map corresponding to the bolt to be tested is determined. The audio feature map is determined based on the audio waveform map corresponding to the bolt to be tested. The audio waveform map is obtained by extracting audio signals from the audio data corresponding to the bolt to be tested. The audio feature map is input into the bolt damage detection model to obtain the damage detection result of the bolt to be detected output by the bolt damage detection model; The bolt damage detection model is determined based on the iterative method of the bolt damage detection model as described in any one of claims 1 to 4.

6. An iterative device for a bolt damage detection model, characterized in that, include: The acquisition unit is used to acquire the target audio data and target damage detection results corresponding to the target bolt. The processing unit is used to extract audio signals from the target audio data to obtain an audio waveform diagram, and to process the audio waveform diagram to obtain an audio feature map. The detection unit is used to input the audio feature map into the initial damage detection model to obtain the predicted damage detection result of the target bolt output by the initial damage detection model; An iterative unit is used to perform parameter iteration on the initial damage detection model based on the target damage detection result and the predicted damage detection result to obtain a bolt damage detection model; The initial damage detection model is used to determine the audio feature vector corresponding to the audio feature map, and to split the audio feature vector. Based on the correlation between the split sub-audio feature vectors and the position feature vectors of the sub-audio feature vectors, the sub-audio feature vectors are fused. Damage detection is performed based on the fused bolt feature vector to obtain the predicted damage detection result of the target bolt. The step of inputting the audio feature map into the initial damage detection model to obtain the predicted damage detection result of the target bolt output by the initial damage detection model includes: The audio feature map is input into the audio feature extraction layer in the initial damage detection model to obtain the sub-audio feature vectors output by the audio feature extraction layer; Each of the sub-audio feature vectors is input into the audio feature fusion layer in the initial damage detection model to obtain the bolt feature vector output by the audio feature fusion layer; The bolt feature vector is input into the damage detection layer in the initial damage detection model to obtain the predicted damage detection result output by the damage detection layer. The audio feature extraction layer is built on top of a residual convolutional network; The audio feature fusion layer is built on top of a self-attention network; The damage detection layer is constructed based on a Gaussian capsule network. The Gaussian capsule network is a capsule network with Gaussian functions as network nodes.

7. A bolt damage detection device, characterized in that, include: The feature map determination unit is used to determine the audio feature map corresponding to the bolt to be detected. The audio feature map is determined based on the audio waveform map corresponding to the bolt to be detected. The audio waveform map is obtained by extracting audio signals based on the audio data corresponding to the bolt to be detected. The damage detection unit is used to input the audio feature map into the bolt damage detection model to obtain the damage detection result of the bolt to be detected output by the bolt damage detection model; the bolt damage detection model is determined based on the iterative method of the bolt damage detection model as described in any one of claims 1 to 4.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the iterative method of the bolt damage detection model as described in any one of claims 1 to 4, or the bolt damage detection method as described in claim 5.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the iterative method of the bolt damage detection model as described in any one of claims 1 to 4, or the bolt damage detection method as described in claim 5.

Images