Iron tower bolt abnormity monitoring method and system based on deep learning and voiceprint

Abstract

The invention discloses an iron tower bolt abnormity monitoring method and system based on deep learning and voiceprint, and the method comprises the steps: firstly obtaining an original monitoring audio of an iron tower, and carrying out the processing of the audio through a pre-trained denoising model, so as to obtain a clearer target audio; and then, extracting a to-be-analyzed bolt voiceprint from the target audio, and loading the to-be-analyzed bolt voiceprint to a pre-trained bolt voiceprint anomaly classification model for classification processing. Finally, the method can accurately obtain the classification result of the bolt voiceprint abnormity in the iron tower monitoring audio, and achieves the intelligent and efficient monitoring of the state of the iron tower bolt.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method and system for monitoring abnormalities of tower bolts based on deep learning and voiceprint. Background Art

[0002] With the continuous advancement of infrastructure construction, iron towers, as key supporting structures, are widely used in the fields of communications, electricity, etc. The stability and safety of iron towers are of vital importance, and bolts, as key components connecting the various parts of the iron tower, are directly related to the overall stability of the iron tower. However, traditional bolt status monitoring methods often rely on manual inspections, which is inefficient and easily affected by human factors.

[0003] Prior art CN118503671A discloses a method for locating and maintaining the loose position of a tower bolt based on deep learning and voiceprint, including: first, obtaining the current feedback voiceprint data of the tower bolt through a preset sensor. Then, calling a pre-trained deep learning model to process the voiceprint data to obtain the loose position of the current tower bolt. Next, determine the target tower bolt corresponding to the current loose position, and obtain the feedback voiceprint data of the target bolt during the overall monitoring cycle, so as to calculate the structural damage risk coefficient of the tower structure where the target bolt is located. Finally, based on the structural damage risk coefficient, determine the maintenance strategy for the target tower bolt. However, in terms of the processing and feature extraction of voiceprint data in the above-mentioned prior patent application scheme, the processing steps of the prior art are relatively traditional and conventional, and may not be able to fully explore the deep features and potential information in the voiceprint data, affecting the accuracy and reliability of the model; a series of complex calculations and conversions are required for the feedback voiceprint data to obtain the structural damage risk coefficient, and the model is complex and the generalization ability is not strong. Summary of the invention

[0004] The purpose of the present invention is to provide a method and system for monitoring abnormalities of tower bolts based on deep learning and voiceprint.

[0005] In a first aspect, an embodiment of the present invention provides a method for monitoring abnormalities of tower bolts based on deep learning and voiceprint, comprising: Get the original tower monitoring audio; Calling a pre-trained tower monitoring audio denoising model to process the original tower monitoring audio to obtain a target tower monitoring audio; Obtaining the bolt soundprint to be analyzed of the target tower monitoring audio; The bolt voiceprint to be analyzed is loaded into a pre-trained bolt voiceprint anomaly classification model for voiceprint classification processing to obtain a target voiceprint anomaly classification result of the target tower monitoring audio.

[0006] In a second aspect, an embodiment of the present invention provides a server system, including a server, wherein the server is used to execute the method described in the first aspect.

[0007] Compared with the prior art, the beneficial effects provided by the present invention include: adopting a method and system for monitoring abnormalities of bolts of iron towers based on deep learning and voiceprints disclosed by the present invention, obtaining the original monitoring audio of the iron tower, and processing the audio using a pre-trained denoising model to obtain a clearer target audio. Subsequently, the bolt voiceprint to be analyzed is extracted from the target audio, and loaded into a pre-trained bolt voiceprint abnormality classification model for classification processing. Ultimately, the method can accurately obtain the bolt voiceprint abnormality classification results in the iron tower monitoring audio, and realize intelligent and efficient monitoring of the status of the iron tower bolts. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only illustrate certain embodiments of the present invention and should not be regarded as limiting the scope. For those of ordinary skill in the art, other related drawings can also be obtained based on these drawings without creative work.

[0009] Figure 1 A schematic diagram of the steps of a method for monitoring abnormalities of tower bolts based on deep learning and voiceprint provided in an embodiment of the present invention; DETAILED DESCRIPTION

[0010] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0011] The specific implementation modes of the present invention are described in detail below in conjunction with the accompanying drawings.

[0012] In order to solve the technical problems in the aforementioned background technology, Figure 1 A flow chart of a method for monitoring abnormalities of iron tower bolts based on deep learning and voiceprints is provided in an embodiment of the present disclosure. The method for monitoring abnormalities of iron tower bolts based on deep learning and voiceprints is introduced in detail below.

[0013] Step S201, obtaining original tower monitoring audio; Step S202, calling a pre-trained tower monitoring audio denoising model to process the original tower monitoring audio to obtain a target tower monitoring audio; Step S203, obtaining the bolt soundprint to be analyzed of the target tower monitoring audio; Step S204, loading the bolt voiceprint to be analyzed into a pre-trained bolt voiceprint anomaly classification model for voiceprint classification processing, and obtaining a target voiceprint anomaly classification result of the target tower monitoring audio.

[0014] In an embodiment of the present invention, illustratively, the server receives the original monitoring audio data from the audio acquisition device installed on the tower. For example, in a certain monitoring task, the server automatically downloads and saves the tower environment audio file of the day from the audio acquisition device at a fixed time every day, such as 3 am, and the file name is "20230425_tower_audio.wav". After obtaining the original audio file, the server automatically calls a pre-trained audio denoising model. This model has been trained with a large amount of tower audio data containing environmental noise, and can effectively identify and remove wind noise, current noise and other background noise. For example, the server denoises the "20230425_tower_audio.wav" file and outputs the denoised audio file "20230425_tower_audio_denoised.wav". The server then runs a voiceprint extraction algorithm to extract the unique voiceprint features of the bolt from the denoised audio. These features may include sound waveforms of specific frequencies, amplitudes, and durations generated when the bolts are loose. For example, from the "20230425_tower_audio_denoised.wav" file, the server successfully extracted a piece of soundprint feature data suspected of bolt loosening and saved it as "20230425_bolt_soundprint.dat". Finally, the server loads the extracted bolt soundprint data into a trained bolt soundprint anomaly classification model. This model can identify normal and abnormal soundprints of bolts, and classify abnormal soundprints in more detail, such as loose bolts, broken bolts, etc. In an embodiment of the present invention, the bolt soundprint anomaly classification model analyzes the "20230425_bolt_soundprint.dat" file and outputs the classification result: "bolt loosening". Corresponding response operations are set in advance in the server for different classification results. According to the result of "bolt loosening", the server can automatically issue an alarm to notify maintenance personnel to check and tighten the bolts on the tower in time. In an embodiment of the present invention, the bolt soundprint anomaly classification model is obtained in the following way.

[0015] Obtaining a current bolt voiceprint instance of a target tower monitoring audio instance; the current bolt voiceprint instance is configured with a current voiceprint abnormality target value; According to the basic voiceprint classification model, an integrated voiceprint classification model and a target voiceprint classification model are constructed; the basic voiceprint classification model is obtained by loading the basic bolt voiceprint instance of the target tower monitoring audio instance into a preset classification model for bolt voiceprint anomaly classification training; the basic bolt voiceprint instance is configured with an initial target tower monitoring audio instance category label; the initial target tower monitoring audio instance category label is different from the current voiceprint anomaly target value; Loading the current bolt voiceprint instance into the integrated voiceprint classification model for voiceprint classification processing to obtain a first sample voiceprint type recognition result; Integrate the first sample voiceprint type recognition result and the current voiceprint abnormality target value to obtain a sample voiceprint abnormality target value; Loading the current bolt voiceprint instance into the target voiceprint classification model for voiceprint classification processing to obtain a second sample voiceprint type recognition result; According to the deviation between the second sample voiceprint type identification result and the sample voiceprint abnormal target value, a training process is performed on the target voiceprint classification model. The trained target voiceprint classification model is used to determine the bolt voiceprint abnormal classification model. The bolt voiceprint abnormal classification model is used to determine the type of the current voiceprint abnormal target value for the bolt voiceprint instance.

[0016] In an embodiment of the present invention, exemplarily, the server selects a specific audio instance, such as "20230425_tower_audio_sample.wav", from the processed tower monitoring audio database, and extracts the current bolt soundprint instance from it, marked as "bolt_soundprint_sample". This bolt soundprint instance is configured with a current soundprint abnormality target value, such as "bolt loosening". The server first uses a preset classification model to train a large number of basic bolt soundprint instances, which are all configured with initial target tower monitoring audio instance category labels, such as "normal", "slightly loose", etc., which are different from the current soundprint abnormality target value "bolt loosening". Through this process, the server obtains a basic soundprint classification model. Then, based on this basic model, the server constructs an integrated soundprint classification model and a target soundprint classification model. The server loads "bolt_soundprint_sample" into the integrated soundprint classification model for processing, and obtains the first sample soundprint type recognition result, for example, the model determines that the soundprint is "possibly loose". The server integrates the recognition result of "possibly loose" with the current voiceprint abnormality target value "loose bolts" to obtain a sample voiceprint abnormality target value, such as "highly suspected of looseness". Next, the server loads "bolt_soundprint_sample" into the target voiceprint classification model for processing, and obtains the second sample voiceprint type recognition result, for example, the model judges the voiceprint to be "slightly loose". The server compares the deviation between "slightly loose" and "highly suspected of looseness", and trains and adjusts the target voiceprint classification model accordingly to improve its recognition accuracy of the degree of bolt looseness. After multiple iterations of training and optimization, the server obtained a more accurate bolt voiceprint abnormality classification model. This model can make more accurate type determination for the current voiceprint abnormality target value (such as "loose bolts") of the bolt voiceprint instance.

[0017] In the embodiment of the present invention, after obtaining the current bolt voiceprint instance of the target tower monitoring audio instance, the embodiment of the present invention also provides the following implementation method.

[0018] Performing data cleaning processing on the current bolt voiceprint instance to obtain a transition bolt voiceprint instance, wherein the data cleaning processing is used to change the composition of the current bolt voiceprint instance; Loading the transition bolt voiceprint instance into the integrated voiceprint classification model and the target voiceprint classification model respectively to obtain a first error parameter; The step of performing a training process on the target voiceprint classification model according to the deviation between the second sample voiceprint type recognition result and the sample voiceprint abnormality target value includes: determining a second error parameter according to a deviation between the second sample voiceprint type recognition result and the sample voiceprint abnormality target value; Determining a target error parameter according to the first error parameter and the second error parameter; According to the target error parameter, the structural parameters of the target voiceprint classification model are adjusted.

[0019] In an embodiment of the present invention, exemplarily, after obtaining the current bolt soundprint instance "bolt_soundprint_sample", the server first performs data cleaning processing on it. For example, the server removes background noise, standardizes volume, trims silent parts, etc. to change the composition of the soundprint instance, and obtains a cleaned transition bolt soundprint instance, named "cleaned_bolt_soundprint_sample". The server then loads "cleaned_bolt_soundprint_sample" into the integrated soundprint classification model and the target soundprint classification model for processing. Both models will output the classification results for the soundprint instance. The server compares the classification results of "cleaned_bolt_soundprint_sample" by the integrated soundprint classification model and the target soundprint classification model, calculates the difference between the two, and obtains a first error parameter. For example, if the integrated model identifies it as "possibly loose" and the target model identifies it as "slightly loose", the first error parameter reflects the difference between the two classifications. The server determines a second error parameter based on the deviation between the second sample voiceprint type recognition result (e.g., "slightly loose") and the sample voiceprint anomaly target value (e.g., "highly suspected loose"). This parameter reflects the accuracy of the target voiceprint classification model when processing a specific voiceprint instance. The server combines the first error parameter and the second error parameter to determine a comprehensive target error parameter. This target error parameter more comprehensively reflects the performance of the target voiceprint classification model when processing voiceprint instances such as "cleaned_bolt_soundprint_sample". Based on the target error parameter, the server adjusts the structural parameters of the target voiceprint classification model. For example, the server adjusts the model's weights, biases, or other hyperparameters to improve the model's classification accuracy for future bolt voiceprint instances. Through this iterative optimization process, the server can continuously improve the performance of the bolt voiceprint anomaly classification model.

[0020] In the embodiment of the present invention, the transition bolt voiceprint instance is loaded into the integrated voiceprint classification model and the target voiceprint classification model respectively to obtain the first error parameter, which can be implemented through the following example.

[0021] Loading the transition bolt voiceprint instance into the integrated voiceprint classification model for voiceprint classification processing to obtain a third sample voiceprint type recognition result; Loading the transition bolt voiceprint instance into the target voiceprint classification model for voiceprint classification processing to obtain a fourth sample voiceprint type recognition result; The first error parameter is determined according to a deviation between the fourth sample voiceprint type recognition result and the third sample voiceprint type recognition result.

[0022] In an embodiment of the present invention, exemplarily, the server loads the "cleaned_bolt_soundprint_sample" transition bolt soundprint instance that has undergone data cleaning processing into the integrated soundprint classification model. This model is composed of multiple basic classifiers, which can comprehensively analyze the input soundprint features and give a classification result. For example, after the integrated soundprint classification model analyzes "cleaned_bolt_soundprint_sample", the third sample soundprint type recognition result given is "suspected medium looseness". Subsequently, the server loads the same "cleaned_bolt_soundprint_sample" transition bolt soundprint instance into the target soundprint classification model. This model is a classifier specifically used to accurately identify abnormalities in specific bolt soundprints. For example, after the target soundprint classification model analyzes "cleaned_bolt_soundprint_sample", the fourth sample soundprint type recognition result given is "moderate looseness". The server will then compare the recognition results given by the integrated soundprint classification model and the target soundprint classification model, namely "suspected medium looseness" and "moderate looseness", and calculate the deviation between the two. This deviation reflects the classification consistency of the two models when processing the same input. Based on this deviation, the server determines the first error parameter. For example, if the two recognition results are completely consistent, the first error parameter may be zero; if there is a difference, the first error parameter will be a positive value used to quantify the degree of this difference. This error parameter will be used in the subsequent model optimization and adjustment process to improve the accuracy and reliability of bolt voiceprint anomaly classification.

[0023] In the embodiments of the present invention, the following implementation modes are also provided.

[0024] When the training cycle is the first preset cycle, the integrated voiceprint classification model is adjusted according to the structural parameters of the target voiceprint classification model to obtain the bolt voiceprint anomaly classification model. The training cycle is the cycle for training the target voiceprint classification model using the current bolt voiceprint instance.

[0025] In the embodiment of the present invention, exemplarily, when the server uses the current bolt soundprint instance "bolt_soundprint_sample" to train the target soundprint classification model, it will set a training cycle. This cycle is called the first preset cycle, which determines the duration and intensity of the model training. For example, the server sets the first preset cycle to 100 training iterations. In each iteration, the server uses "bolt_soundprint_sample" and its related soundprint abnormality target value to train the target soundprint classification model, and continuously adjusts the parameters of the model to improve its recognition ability of bolt soundprint abnormalities. When the training reaches the first preset cycle, that is, after 100 iterations, the server adjusts the integrated soundprint classification model according to the current structural parameters of the target soundprint classification model. Specifically, the server analyzes the features and classification logic learned by the target soundprint classification model during the training process, and integrates this information into the integrated soundprint classification model. For example, if the target soundprint classification model finds that certain specific frequency components in "bolt_soundprint_sample" are essential for identifying bolt looseness during the training process, the server will add these features to the recognition logic of the integrated soundprint classification model. Through such adjustments, the integrated voiceprint classification model can absorb the learning results of the target voiceprint classification model in a specific training cycle, thereby improving its recognition performance and generalization ability for bolt voiceprint anomalies. Ultimately, the adjusted integrated voiceprint classification model will be used as a bolt voiceprint anomaly classification model to more accurately identify and process bolt voiceprint anomalies in tower monitoring audio.

[0026] In the embodiment of the present invention, when the training cycle is the first preset cycle, the integrated voiceprint classification model is adjusted according to the structural parameters of the target voiceprint classification model to obtain the bolt voiceprint anomaly classification model, which can be implemented through the following examples.

[0027] Adjusting the integrated voiceprint classification model according to the structural parameters of the target voiceprint classification model to obtain a candidate integrated voiceprint classification model; During the training process when the training cycle is the first preset cycle, the candidate integrated voiceprint classification model is tuned according to the structural parameters of the target voiceprint classification model to obtain the bolt voiceprint anomaly classification model.

[0028] In an embodiment of the present invention, exemplarily, the server first analyzes the structural parameters of the target voiceprint classification model after the first preset cycle of training. These structural parameters may include the weights, biases, connection methods between layers, and other hyperparameters of the model. For example, the target voiceprint classification model can learn some feature extraction methods and classification boundaries that are particularly effective for identifying bolt loosening soundprints. Based on these analyses, the server adjusts the integrated voiceprint classification model. The adjustment methods may include adding or deleting certain feature extraction layers, modifying connection weights, adjusting classification thresholds, etc. These adjustments are intended to enable the integrated voiceprint classification model to absorb the advantages of the target voiceprint classification model. After these adjustments, the server obtains a candidate integrated voiceprint classification model. In the subsequent training process, the server will continue to use bolt voiceprint instances such as "bolt_soundprint_sample" to tune the candidate integrated voiceprint classification model. This tuning process may include multiple aspects, such as further adjusting the weights and biases of the model, optimizing the layer structure of the model, improving the activation function, etc. For example, the server may find that the candidate integrated voiceprint classification model has misjudgments when identifying certain specific types of bolt loosening soundprints. To address this problem, the server can fine-tune the relevant layers in the model to improve the model's recognition accuracy for this type of voiceprint. After multiple iterations of tuning in the first preset cycle, the server finally obtained a bolt voiceprint anomaly classification model with better performance. This model not only inherits the advantages of the target voiceprint classification model, but also improves generalization ability and stability through ensemble learning. In this way, the server can use this model to more accurately identify bolt voiceprint anomalies in the tower monitoring audio.

[0029] In an embodiment of the present invention, during the training process when the training cycle is the first preset cycle, the candidate integrated voiceprint classification model is tuned according to the structural parameters of the target voiceprint classification model to obtain the bolt voiceprint anomaly classification model, which can be implemented through the following examples.

[0030] When the training cycle is the first preset cycle, after training for a training cycle of the second preset cycle, adjusting the candidate integrated voiceprint classification model according to the structural parameters of the target voiceprint classification model; When the preset training termination condition is reached, the candidate integrated voiceprint classification model is determined as the bolt voiceprint anomaly classification model.

[0031] In an embodiment of the present invention, exemplarily, during the model training process of the server, when the training reaches the first preset cycle, a new training phase will be entered, which is called the second preset cycle. In this phase, the server will continue to pay attention to the structural parameters and performance of the target voiceprint classification model, and further adjust the candidate integrated voiceprint classification model based on this. For example, after the first preset cycle, the target voiceprint classification model shows a high recognition rate for bolt loosening soundprints of certain specific frequencies. After analyzing the structural parameters of this target voiceprint classification model, the server found that it used a special feature extraction method. Therefore, during the training process of the second preset cycle, the server will apply this method to the candidate integrated voiceprint classification model to improve its recognition ability for bolt loosening soundprints of specific frequencies. At the same time, the server will also optimize and adjust the candidate integrated voiceprint classification model according to other excellent characteristics of the target voiceprint classification model, such as noise resistance, recognition speed, etc. During the training process of the second preset cycle, the server will continue to monitor the performance of the candidate integrated voiceprint classification model and compare it with the preset training termination condition. These training termination conditions may include the recognition accuracy of the model, the training duration, the convergence of the loss function, etc. For example, the server sets that when the recognition accuracy of the candidate integrated voiceprint classification model no longer significantly improves in multiple consecutive training iterations, or when the training duration reaches a preset upper limit, the training termination condition is considered to have been met. At this point, the server will determine the current candidate integrated voiceprint classification model as the final bolt voiceprint anomaly classification model. This final bolt voiceprint anomaly classification model not only inherits the advantages of the target voiceprint classification model, but also improves the recognition performance and stability through integrated learning and multi-cycle training tuning. The server can use this model to more accurately identify and classify bolt voiceprint anomalies in the tower monitoring audio.

[0032] In the embodiment of the present invention, when the training cycle of the second preset cycle is completed, the candidate integrated voiceprint classification model is adjusted according to the structural parameters of the target voiceprint classification model, which can be implemented through the following examples.

[0033] After training for a second preset training period, obtaining current structural parameters of the target voiceprint classification model; Processing the current structural parameters according to the state space model to obtain target structural parameters; The candidate integrated voiceprint classification model is adjusted according to the target structure parameters.

[0034] In the embodiment of the present invention, exemplarily, during the training process of the second preset cycle, the server will first obtain the current structural parameters of the target voiceprint classification model. These structural parameters may include the weight matrix, bias vector, convolution kernel parameters, etc. of the model, which together determine the recognition performance and feature extraction capability of the model. For example, the server can extract these structural parameters by accessing the internal state of the target voiceprint classification model or using special tools. For example, the server obtains a set of weight matrices and bias vectors, which represent the feature transformation and decision boundaries of the model at different levels. Next, the server will use a state space model to process the currently obtained structural parameters. The state space model can help the server understand and predict the change law of the model parameters over time, so as to extract more stable and representative target structural parameters. For example, the server can construct a Kalman filter or a hidden Markov model as a state space model, and use these models to filter, smooth or predict the current structural parameters. Through the processing of the state space model, the server can remove noise and outliers and obtain more robust and reliable target structural parameters. Finally, the server will adjust the candidate integrated voiceprint classification model according to the obtained target structural parameters. The purpose of the adjustment is to enable the candidate integrated voiceprint classification model to better learn and simulate the excellent characteristics of the target voiceprint classification model, thereby improving its own recognition performance and generalization ability. For example, the server can apply the weight matrix and bias vector in the target structural parameters to the corresponding level of the candidate integrated voiceprint classification model, or redesign the network structure and connection mode of the candidate integrated voiceprint classification model according to the target structural parameters. Through these adjustments, the candidate integrated voiceprint classification model can more effectively extract the characteristics of the bolt voiceprint and make accurate classification judgments. In summary, by obtaining the current structural parameters of the target voiceprint classification model, processing using the state space model, and adjusting the candidate integrated voiceprint classification model according to the target structural parameters, the server can continuously improve the performance of the bolt voiceprint anomaly classification model to more accurately identify and process the bolt voiceprint anomalies in the tower monitoring audio.

[0035] In the embodiment of the present invention, the current structural parameters are processed according to the state space model to obtain the target structural parameters, which can be implemented through the following examples.

[0036] Determine, according to the training cycle corresponding to the current structural parameter, a first influencing factor of the integrated voiceprint classification model and a second influencing factor of the target voiceprint classification model; the first influencing factor has a negative feedback relationship with the training cycle, and the second influencing factor has a positive feedback relationship with the training cycle; The current structural parameters are processed according to the first influencing factor, the second influencing factor and the state space model to obtain the target structural parameters.

[0037] In the embodiment of the present invention, exemplarily, during the model training process of the server, two influencing factors are determined for each training cycle: the first influencing factor of the integrated voiceprint classification model and the second influencing factor of the target voiceprint classification model. These two influencing factors are used to adjust the weight of the model parameter update to ensure the stability and convergence of the training process. For example, the current training cycle is the 10th cycle of the second preset cycle, and the server determines the first influencing factor and the second influencing factor according to a preset functional relationship. Since the first influencing factor has a negative feedback relationship with the training cycle, the value of the first influencing factor will gradually decrease as the training cycle increases. On the contrary, the second influencing factor has a positive feedback relationship with the training cycle, and its value will gradually increase as the training cycle increases. Specifically, if the initial first influencing factor is set to 0.8 and the second influencing factor is set to 0.2, as the training cycle progresses, the first influencing factor will gradually decrease to 0.6, 0.4, etc., and the second influencing factor will gradually increase to 0.4, 0.6, etc. After determining the first influencing factor and the second influencing factor, the server will use these two influencing factors and the state space model to process the current structural parameters. The state space model can help the server predict and estimate the optimal values ​​of the model parameters based on historical and current information. For example, the server uses a Kalman filter as a state space model, which can update the state of the model by combining the currently observed structural parameters and the previous estimated values. During the update process, the first influencing factor acts on the original parameters of the integrated voiceprint classification model, gradually reducing its influence; while the second influencing factor enhances the influence of the current structural parameters of the target voiceprint classification model. In this way, the server can gradually integrate the excellent characteristics of the target voiceprint classification model into the integrated voiceprint classification model while maintaining the stability and generalization ability of the integrated model. Ultimately, the target structural parameters obtained after processing will be a parameter set that combines the advantages of multiple models, which is used to guide the subsequent training and tuning process. In summary, by determining the first influencing factor and the second influencing factor, and using these influencing factors and the state space model to process the current structural parameters, the server can more effectively perform model fusion and parameter optimization, thereby improving the performance of the bolt voiceprint anomaly classification model.

[0038] In an embodiment of the present invention, the training method of the basic voiceprint classification model can be implemented through the following examples.

[0039] Obtain the foundation bolt voiceprint instance of the target tower monitoring audio instance; Loading the basic bolt voiceprint instance into the preset classification model for voiceprint classification processing to obtain an initial sample voiceprint type recognition result; Determining a sample error parameter according to a deviation between the initial sample voiceprint type recognition result and the initial target tower monitoring audio instance category label; The structural parameters of the preset classification model are adjusted according to the sample error parameters, and the preset classification model that has completed training is determined as the basic voiceprint classification model.

[0040] In an embodiment of the present invention, illustratively, the server first extracts the basic bolt soundprint instance from the target tower monitoring audio instance. These instances may contain soundprint features of bolt loosening, breaking or other abnormal situations. The server can intercept these representative soundprint fragments through professional audio processing software or custom algorithms. For example, the server can accurately intercept the sound wave fragment of a specific frequency and amplitude generated when the bolt is loose from a long period of tower monitoring audio, and this fragment is the basic bolt soundprint instance. Next, the server will load the extracted basic bolt soundprint instance into a preset classification model for soundprint classification processing. This preset classification model can be a deep learning network, such as a convolutional neural network (CNN) or a recurrent neural network (RNN), which has been preliminarily trained for audio classification tasks. For example, the server inputs the intercepted bolt loose soundprint instance into the preset CNN model, and the model will try to identify which type (such as loose, broken, etc.) this soundprint instance belongs to, and output an initial sample soundprint type recognition result. After obtaining the initial sample soundprint type recognition result, the server will compare it with the category label of the initial target tower monitoring audio instance to calculate the deviation between the two. This deviation is called the sample error parameter, which reflects the degree of inconsistency between the model prediction result and the actual label. For example, if the model mistakenly identifies the voiceprint instance of loose bolts as broken bolts, the sample error parameter will be relatively large. According to the calculated sample error parameter, the server will adjust the structural parameters of the preset classification model to reduce the error and improve the recognition accuracy of the model. The adjustment method can include modifying the network weight, increasing or decreasing the number of network layers, changing the activation function, etc. For example, the server will update the weight matrix of the CNN model through the back propagation algorithm so that the model can more accurately identify the next time it encounters a similar bolt loose voiceprint instance. After multiple iterations of training and adjustment, when the performance of the preset classification model reaches a certain preset standard (such as the recognition accuracy exceeds 90%), the server will determine it as the final basic voiceprint classification model. This model will be used in subsequent bolt voiceprint anomaly classification tasks. In summary, by obtaining basic bolt voiceprint instances, loading them into preset classification models for processing, determining sample error parameters, and adjusting model structure parameters, the server can train a high-performance basic voiceprint classification model to provide strong support for subsequent tower bolt voiceprint anomaly classification.

[0041] In addition, in the “Category Labels of Initial Target Tower Detection Audio Instances”, defining or labeling the category labels of voiceprint instances in different situations is a key preprocessing step, which involves corresponding audio instances to specific physical states or abnormal conditions. First, it is necessary to clearly define various possible bolt states or abnormal conditions, such as “loose”, “broken”, “normal”, etc., and assign a unique category label to each state. These labels should be clear, consistent, and easy to understand for use in subsequent model training and classification tasks. For each tower monitoring audio instance, it is necessary to be carefully listened to by professional technicians or trained annotators to determine the specific state of the bolts in the audio. The annotator will label each audio instance as the corresponding bolt state, such as “loose” or “broken”, based on the predefined category labels. In order to ensure the accuracy of the labeling, multiple labeling and cross-validation strategies can be implemented. That is, each audio instance can be independently labeled by multiple annotators, and then their labeling results can be compared. If the labeling results are inconsistent, a higher-level technician can arbitrate to determine the final category label. After the labeling is completed, the audio instance and its corresponding category label need to be stored in the database for subsequent model training and classification tasks. When storing data, ensure that the association between each audio instance and its category label is clear and unambiguous. As time goes by and experience accumulates, the definition of category labels may need to be updated or adjusted. For example, if a new abnormal state of a bolt is found, a new category label can be assigned to it. At the same time, it is also necessary to regularly review and verify the existing labeled data to ensure its accuracy and consistency. Through the above process, the accuracy and consistency of the "category label of the initial target tower detection audio instance" can be ensured, providing reliable data support for the subsequent basic voiceprint classification model training.

[0042] In an embodiment of the present invention, the tower monitoring audio denoising model is obtained in the following manner: obtaining a noise-free tower monitoring audio instance, a noisy tower monitoring audio instance matched by the noise-free tower monitoring audio instance, and a noise label of the noisy tower monitoring audio instance; the noise label indicates the noise data in the noisy tower monitoring audio instance; Using the first noise elimination model, performing a noise filtering operation on the noisy iron tower monitoring audio instance to determine the first iron tower monitoring filtered audio, performing noise data estimation on the noisy iron tower monitoring audio instance to determine the estimated noise data information of the noisy iron tower monitoring audio instance; According to the estimated noise data information, the bolt audio data in the noisy iron tower monitoring audio instance is obtained, and from the first iron tower monitoring filtered audio, the audio data having the same timing node as the bolt audio data in the noisy iron tower monitoring audio instance is collected, and the bolt audio data in the noisy iron tower monitoring audio instance is used to replace the audio data in the first iron tower monitoring filtered audio, so as to determine the iron tower monitoring filtered audio instance; Obtaining a noise filtering cost parameter of the first noise elimination model according to the tower monitoring filtered audio instance and the noise-free tower monitoring audio instance; Obtaining a noise data estimation cost parameter of the first noise elimination model according to the estimated noise data information and the noise label; According to the noise filtering cost parameter and the noise data estimation cost parameter, a tuning operation is performed on the structural parameters of the first noise elimination model to determine a tower monitoring audio denoising model for performing a noise filtering operation on the original tower monitoring audio.

[0043] In an embodiment of the present invention, illustratively, the server first obtains noise-free tower monitoring audio instances, noisy tower monitoring audio instances matching these noise-free audio instances, and noise labels corresponding to these noisy audio instances from the tower monitoring system. The noise label clearly indicates which parts of the audio are noise data. For example, the server can record noise-free bolt tightening sounds from a stable tower monitoring environment as noise-free audio instances, and record the sound of the same operation from another tower monitoring environment disturbed by the environment (such as wind noise, traffic noise, etc.) as a noisy audio instance. The noise label is annotated by an expert, and clearly indicates which time periods are noise in the noisy audio. The server will then use an initial noise elimination model (called the first noise elimination model) to perform noise filtering operations on the noisy tower monitoring audio instances to generate the first tower monitoring filtered audio. At the same time, this model will also estimate the noise data in the noisy audio to generate estimated noise data information. For example, the first noise elimination model can be a noise reduction model based on deep learning, which can identify and remove the noise part in the audio, and at the same time estimate the specific information of the removed noise, such as the type and intensity of the noise. The server extracts the bolt audio data (i.e., the valid audio data after noise removal) from the noisy audio based on the estimated noise data information, and collects the audio data of the same time series nodes as these bolt audio data from the first tower monitoring filtered audio. Then, the server replaces the corresponding audio data in the first tower monitoring filtered audio with the bolt audio data in the noisy audio, thereby generating a tower monitoring filtered audio instance. For example, if the estimated noise data information shows that there is strong traffic noise in a certain period of time, the server extracts the bolt tightening sound of the non-noise part of this period from the noisy audio, and replaces the audio data of the same period in the first tower monitoring filtered audio with this part of the sound. The server then calculates the noise filtering cost parameter of the first noise elimination model based on the tower monitoring filtered audio instance and the noise-free tower monitoring audio instance. This cost parameter reflects the performance of the model in noise filtering. At the same time, based on the estimated noise data information and the actual noise label, the server also calculates the noise data estimated cost parameter, which reflects the accuracy of the model in noise estimation. For example, the noise filtering cost parameter can be calculated by comparing the similarity between the filtered audio instance and the noise-free audio instance, while the noise data estimation cost parameter can be calculated by comparing the consistency between the estimated noise data information and the actual noise label. Finally, the server will tune the structural parameters of the first noise elimination model according to the calculated noise filtering cost parameter and the noise data estimation cost parameter to improve the performance of the model in noise filtering and noise estimation. The tuned model will be determined as the final tower monitoring audio denoising model, which is used to perform noise filtering operations on the original tower monitoring audio.For example, if the noise filtering cost parameter shows that the model performs poorly in filtering certain types of noise, the server will adjust the model's weights or network structure to improve performance in this area. Similarly, if the noise data estimation cost parameter shows that the model is biased in estimating noisy data, the server will also adjust the model accordingly to improve the accuracy of the estimation.

[0044] In an embodiment of the present invention, the first noise elimination model includes a feature enhancement module, a noise filtering module and a noise data estimation module; The method of utilizing the first noise elimination model to perform noise filtering operation on the noisy tower monitoring audio instance to determine the first tower monitoring filtered audio, estimating noise data on the noisy tower monitoring audio instance to determine the estimated noise data information of the noisy tower monitoring audio instance can be implemented through the following examples.

[0045] Using the feature enhancement module, feature enhancement is performed on the noisy iron tower monitoring audio instance to determine the feature enhanced audio matched by the noisy iron tower monitoring audio instance; Using the noise filtering module, a noise filtering operation is performed on the feature enhanced audio matched with the noisy iron tower monitoring audio instance to determine the first iron tower monitoring filtered audio; The noise data estimation module is used to perform noise data estimation on the feature-enhanced audio matched with the noisy iron tower monitoring audio instance, and the estimated noise data information of the noisy iron tower monitoring audio instance is determined.

[0046] In an embodiment of the present invention, exemplarily, the server first uses a feature enhancement module to perform feature enhancement on a noisy tower monitoring audio instance. The purpose of feature enhancement is to highlight key information in the audio, especially those parts that may be masked or weakened by noise, so as to help subsequent noise filtering and estimation to be more accurate. For example, the server receives a noisy tower monitoring audio instance containing wind sound and bolt loosening sound. The feature enhancement module will use a series of algorithms (such as spectrum analysis, wavelet transform, etc.) to enhance the characteristics of the bolt loosening sound, while suppressing or weakening the characteristics of noise such as wind sound. In this way, the characteristics of the bolt loosening sound will be more obvious in the audio after feature enhancement (i.e., feature enhanced audio). Next, the server uses a noise filtering module to perform a noise filtering operation on the feature enhanced audio. The purpose of the noise filtering module is to remove or reduce the interference of noise from the audio, so as to obtain a clearer target sound. The noise filtering module will process the feature enhanced audio and identify and remove noise components such as wind sound through advanced signal processing techniques (such as adaptive filtering, spectrum subtraction, etc.). After the noise filtering operation, the server will obtain the first tower monitoring filtered audio, in which the sound of loose bolts is more clearly identifiable, while the noise such as wind is greatly suppressed or eliminated. Finally, the server uses the noise data estimation module to estimate the noise data of the feature-enhanced audio. The purpose of noise data estimation is to analyze and estimate the type, intensity, and distribution of noise in the audio, which is very important for evaluating the effect of noise filtering and subsequent optimization processing. In the above example, the noise data estimation module will conduct an in-depth analysis of the feature-enhanced audio and estimate the noise components through machine learning algorithms or statistical models. For example, it can identify the frequency range, intensity changes, and other characteristics of wind sound, and generate estimated noise data information. This information can not only help the server evaluate the performance of the noise filtering module, but also provide valuable reference for subsequent model tuning and parameter adjustment.

[0047] In an embodiment of the present invention, the feature enhancement module includes a robust scaling submodule and a feature enhancement submodule; the feature enhancement module is used to perform feature enhancement on the noisy tower monitoring audio instance and determine the feature enhanced audio that matches the noisy tower monitoring audio instance, which can be implemented through the following examples.

[0048] Using the robust scaling submodule, the noisy tower monitoring audio instance is robustly scaled according to the audio attribute matched by the noisy tower monitoring audio instance, and a robustly scaled noisy tower monitoring audio instance is determined; The feature enhancement submodule is used to perform feature enhancement on the robustly scaled noisy iron tower monitoring audio instance, and determine feature enhanced audio that matches the noisy iron tower monitoring audio instance.

[0049] In an embodiment of the present invention, exemplarily, the server first uses a robust scaling submodule to perform robust scaling processing on a noisy tower monitoring audio instance. The purpose of this step is to adjust the dynamic range of the audio so that it is more suitable for subsequent feature enhancement processing, while reducing the impact of outliers or noise on the overall data. For example, the server receives a noisy tower monitoring audio instance with large volume fluctuations, in which the volume of some parts is too high and other parts are relatively low. The robust scaling submodule analyzes the audio properties of this audio instance (such as volume, frequency distribution, etc.), and then uses a robust scaling method (such as median absolute deviation scaling, robust standardization, etc.) to adjust the amplitude of the audio so that it can reduce the impact of volume fluctuations on subsequent processing while maintaining the original audio features. The volume and dynamic range of the audio instance after robust scaling processing will be more balanced and consistent. Next, the server uses the feature enhancement submodule to perform feature enhancement on the noisy tower monitoring audio instance after robust scaling. The goal of feature enhancement is to extract and strengthen the key features in the audio so as to better perform subsequent noise filtering and estimation. The feature enhancement submodule will further analyze and process the audio instance after robust scaling processing. It may use a series of signal processing techniques (such as spectrum analysis, time-frequency analysis, etc.) to extract key features in the audio, such as the sound characteristics of loose bolts, the frequency characteristics of wind sound, etc. Then, by enhancing these key features (such as increasing the amplitude of the sound of loose bolts, sharpening its spectral characteristics, etc.), these features are easier to identify and distinguish in subsequent noise filtering and estimation. After being processed by the feature enhancement submodule, the server will obtain feature-enhanced audio that matches the original noisy tower monitoring audio instance, in which the key features are significantly enhanced and highlighted.

[0050] In an embodiment of the present invention, the noise filtering module includes a filtering encoding submodule, a filtering feature mapping module and a filtering reverse robust scaling submodule; The use of the noise filtering module to perform a noise filtering operation on the feature-enhanced audio matched with the noisy tower monitoring audio instance to determine the first tower monitoring filtered audio can be implemented through the following example.

[0051] Using the filtering and coding submodule, feature coding is performed on the feature-enhanced audio matched by the noisy iron tower monitoring audio instance, and a first feature-coded audio matched by the noisy iron tower monitoring audio instance is determined; Using the filtering feature mapping module, the first feature-coded audio is subjected to feature mapping processing to determine the first feature-mapped audio matched by the noisy tower monitoring audio instance; The first feature mapping audio is subjected to reverse robust scaling processing by utilizing the filtering reverse robust scaling submodule to determine the first tower monitoring filtered audio.

[0052] In an embodiment of the present invention, exemplarily, the server first uses the filtering coding submodule to perform feature coding on the feature-enhanced audio matched with the noisy tower monitoring audio instance. The purpose of feature coding is to convert the audio data into a format that is easier to process and analyze while retaining the key features of the audio. For example, the server has obtained a feature-enhanced tower monitoring audio through the feature enhancement module, which contains noises such as loose bolts and wind. The filtering coding submodule will use a suitable encoding method (such as Mel frequency cepstral coefficient MFCC, linear prediction coding LPC, etc.) to encode this audio. The encoded audio (i.e., the first feature-encoded audio) converts the waveform data of the audio into a series of feature vectors, which can more accurately describe the sound characteristics in the audio, such as pitch, timbre, etc., and provide convenience for subsequent feature mapping and noise filtering. Next, the server uses the filtering feature mapping module to perform feature mapping processing on the first feature-encoded audio. The purpose of feature mapping is to map the encoded feature vector to a new feature space, so that in this new space, the target sound (such as the sound of loose bolts) and the noise (such as the sound of wind) can be more easily distinguished. The filtering feature mapping module uses an advanced mapping algorithm (such as autoencoder and principal component analysis in deep learning) to process the first feature encoded audio. Through training and learning, this module can identify the different distribution patterns of noises such as loose bolts and wind noise in the feature space and map them to different areas. In this way, in the audio processed by feature mapping (i.e., the first feature mapping audio), the characteristics of noises such as loose bolts and wind noise will be more clearly separated. Finally, the server uses the filtering reverse robust scaling submodule to perform reverse robust scaling on the first feature mapping audio. The purpose of reverse robust scaling is to restore the audio data processed by feature mapping to the original dynamic range while maintaining the filtering effect on noise. In the above example, the filtering reverse robust scaling submodule uses the reverse operation corresponding to the robust scaling submodule to process the first feature mapping audio. It restores the audio data after feature mapping to a dynamic range close to the original audio based on the dynamic range and scaling factor of the original audio. In this way, in the audio after reverse robust scaling (i.e. the first tower monitoring filtered audio), the sound of bolt loosening will be more clearly identifiable, while noise such as wind noise will be effectively filtered out or its impact will be reduced. At the same time, this filtered audio also retains the dynamic range and volume change characteristics of the original audio, making subsequent audio processing and analysis more accurate and reliable.

[0053] In an embodiment of the present invention, the filtering and encoding submodule includes a plurality of cascaded encoding networks; The use of the filtering and coding submodule to perform feature coding on the feature-enhanced audio matched with the noisy tower monitoring audio instance to determine the first feature-coded audio of the noisy tower monitoring audio instance may be implemented through the following examples.

[0054] The output current intermediate feature audio of the current coding network is input into the subsequent coding network; the input of the initial coding network is the feature enhanced audio matched by the noisy tower monitoring audio instance; Using the post-order coding network, feature coding the output current intermediate feature audio to determine the initial feature coded audio matched by the post-order coding network; The initial feature coded audio matched by the post-order coding network and the output current intermediate feature audio are merged to determine the output intermediate feature audio of the post-order coding network; The output intermediate feature audio of the last encoding network is used as the first feature encoded audio of the noisy tower monitoring audio instance.

[0055] In an embodiment of the present invention, exemplarily, the server first uses the feature-enhanced audio as the input of the initial encoding network. For example, the feature-enhanced audio is an audio file containing the sound of loose bolts and wind, and this file is first sent to the first encoding network. Each encoding network will feature encode the input audio and output an intermediate feature audio. This intermediate feature audio will serve as the input of the next encoding network. For example, the first encoding network will extract the key features in the feature-enhanced audio, such as the frequency and amplitude of the sound of loose bolts, and then output an intermediate feature audio. This intermediate feature audio is then input into the second encoding network. The second encoding network will further extract and encode these features, and then output the second intermediate feature audio. This process will continue until the last encoding network. At each encoding network stage, the server will fuse the initial feature encoding audio matched by the subsequent encoding network and the output current intermediate feature audio. This fusion can be achieved by weighted averaging, superposition or other complex fusion algorithms, in order to better retain and refine the key features in the audio. For example, in the second encoding network stage, the server will fuse the output of the first encoding network (i.e., the first intermediate feature audio) and the initial feature encoded audio of the second encoding network to obtain the second intermediate feature audio. This process will be repeated in each encoding network stage. After the audio data passes through all the cascaded encoding networks, the server will use the output intermediate feature audio of the last encoding network as the first feature encoded audio of the noisy tower monitoring audio instance. This first feature encoded audio is a highly refined and encoded audio representation that contains the key feature information in the original audio, but removes a large amount of redundant and noise data. Through this cascaded encoding network processing process, the server can more accurately extract and encode the key features in the noisy tower monitoring audio instance, providing a more accurate data basis for subsequent feature mapping and noise filtering.

[0056] In an embodiment of the present invention, the noise data estimation module includes an estimation coding submodule, an estimation feature mapping module and an estimation reverse robust scaling submodule; The use of the noise data estimation module to perform noise data estimation on the feature-enhanced audio matched with the noisy tower monitoring audio instance to determine the estimated noise data information of the noisy tower monitoring audio instance can be implemented through the following examples.

[0057] Using the estimation coding submodule, feature coding is performed on the feature-enhanced audio matched by the noisy iron tower monitoring audio instance, and a second feature-coded audio matched by the noisy iron tower monitoring audio instance is determined; Using the estimated feature mapping module, the second feature-encoded audio is subjected to feature mapping processing to determine the second feature-mapped audio matched by the noisy tower monitoring audio instance; Using the estimated reverse robust scaling submodule, the second feature map audio is subjected to reverse robust scaling processing to determine the reverse robustly scaled second feature map audio; According to the second feature mapping audio after the reverse robust scaling, the estimated noise data information of the noisy tower monitoring audio instance is obtained.

[0058] In an embodiment of the present invention, exemplarily, first, the server uses the estimated coding submodule to perform feature coding on the feature-enhanced audio. For example, the feature-enhanced audio contains complex noises such as loose bolts and wind sounds, and the estimated coding submodule uses a specific coding algorithm (such as convolutional neural network CNN in deep learning, etc.) to extract key features in the audio. After encoding, the server obtains a second feature-encoded audio, which represents the feature information in the original audio in a more compact and efficient form. Next, the server uses the estimated feature mapping module to perform feature mapping processing on the second feature-encoded audio. The purpose of this module is to map the encoded features to a new space so that noise and target sounds can be more easily distinguished. For example, through the training and learning of a deep learning model (such as an autoencoder), the estimated feature mapping module can identify the different distribution patterns of noises such as loose bolts and wind sounds in the feature space and map them to different areas. After mapping processing, the server obtains a second feature-mapped audio, in which the features of the noise and the target sound are more clearly separated. Then, the server uses the estimated reverse robust scaling submodule to perform reverse robust scaling processing on the second feature-mapped audio. The purpose of this step is to restore the feature-mapped audio data to the original dynamic range while maintaining the accuracy of the estimated noise data. Through the reverse robust scaling process, the server obtains a second feature-mapped audio after reverse robust scaling, which retains the dynamic range of the original audio while highlighting the characteristics of the noise data. Finally, the server analyzes and determines the estimated noise data information based on the second feature-mapped audio after reverse robust scaling. For example, by calculating parameters such as the energy and frequency distribution of the noise components in the audio, the server can obtain a detailed estimate report on the noise. This report can include key information such as the type, intensity, and duration of the noise, which provides an important basis for subsequent tower maintenance and noise control. Through this process, the server can accurately estimate the noise data information in the noisy tower monitoring audio instance, providing strong support for subsequent noise filtering and tower status assessment.

[0059] In the embodiment of the present invention, obtaining the estimated noise data information of the noisy tower monitoring audio instance according to the second feature mapping audio after the reverse robust scaling can be implemented through the following examples.

[0060] The second feature map audio after the reverse robust scaling is subjected to standardization processing to determine an estimated noise label of the noisy iron tower monitoring audio instance; the estimated noise label represents the noise data in the noisy iron tower monitoring audio instance; The estimated noise label of the noisy iron tower monitoring audio instance is regarded as the estimated noise data information of the noisy iron tower monitoring audio instance.

[0061] In an embodiment of the present invention, exemplarily, the server first performs standardization processing on the second feature map audio after reverse robust scaling. Standardization processing is an important data preprocessing step, which can eliminate the dimensionality effect of the data and make different features comparable. In this scenario, the purpose of standardization processing is to convert the noise data in the second feature map audio after reverse robust scaling into a unified metric for subsequent analysis and comparison. For example, the server can use the Z-score standardization method to convert the noise data in the audio into a distribution with a mean of 0 and a standard deviation of 1. In this way, the noise data is converted to a relatively fixed scale for subsequent processing. After standardization processing, the server determines the estimated noise label of the noisy tower monitoring audio instance based on the processed noise data. This noise label is actually an identifier used to characterize the noise data in the audio. It can be a simple binary classification label (such as "noise" or "no noise"), or it can be a more complex label containing information such as the type and intensity of the noise. For example, the server can set a threshold, and when the standardized noise data exceeds this threshold, it is marked as "noise", otherwise it is marked as "noise". Alternatively, the server can also use more complex machine learning algorithms to classify and label the standardized noise data. Finally, the server uses the estimated noise label of the noisy tower monitoring audio instance as the estimated noise data information of the audio instance. This information can be stored in a database or sent to other systems or users through a network for subsequent processing and analysis. For example, the server can associate the estimated noise label with the original audio file and store it in the same database. In this way, when the user needs to query the noise condition of a certain audio file, the server can directly provide the estimated noise label of this file as a reference. Through this process, the server can accurately extract and label the noise data in the noisy tower monitoring audio instance, providing an important basis for subsequent tower status assessment and noise control.

[0062] In the embodiment of the present invention, the step of obtaining the bolt audio data in the noisy tower monitoring audio instance according to the estimated noise data information may be implemented through the following example.

[0063] According to the estimated noise data information, noise data in the noisy tower monitoring audio instance is obtained; The audio data other than the noise data in the noisy iron tower monitoring audio instance is regarded as the bolt audio data in the noisy iron tower monitoring audio instance.

[0064] In an embodiment of the present invention, illustratively, the server first identifies and extracts the noise data in the noisy tower monitoring audio instance based on the estimated noise data information. The estimated noise data information may include the type of noise, timestamp, frequency range, etc., which helps the server to accurately locate and extract the noise part in the audio. For example, if the estimated noise data information indicates that there is wind noise between the 10th and 20th seconds of the audio, the server will extract the data of this time period from the original audio and mark it as noise data. After identifying and extracting the noise data, the server will treat the audio data other than the noise data in the noisy tower monitoring audio instance as bolt audio data. This part of the data is considered to be sound related to the bolt state because they are not identified as noise. For example, if the total length of the original audio is 30 seconds, and the server has extracted the wind noise between the 10th and 20th seconds, then the remaining audio data from the 1st to the 10th second and the 20th to the 30th second will be regarded as bolt audio data. This process is actually a data cleaning process. By removing noise data, the server retains the audio information directly related to the bolt status, providing a more accurate data basis for subsequent bolt status analysis. In summary, through these two steps, the server can use the estimated noise data information to extract the bolt audio data from the noisy tower monitoring audio, providing important reference information for the maintenance and inspection of the tower.

[0065] In an embodiment of the present invention, the structural parameters of the first noise elimination model are tuned according to the noise filtering cost parameter and the noise data estimation cost parameter to determine a tower monitoring audio denoising model for performing a noise filtering operation on the original tower monitoring audio, which can be implemented through the following example.

[0066] Obtaining the contribution of the noise filtering cost parameter and the noise data estimation cost parameter respectively matching each other; According to the contribution degrees of the noise filtering cost parameter and the noise data estimation cost parameter respectively matched, multi-objective optimization processing is performed on the noise filtering cost parameter and the noise data estimation cost parameter to determine the overall cost parameter of the first noise elimination model; According to the overall cost parameter of the first noise elimination model, a tuning operation is performed on the structural parameters of the first noise elimination model to determine a tower monitoring audio denoising model for performing a noise filtering operation on the original tower monitoring audio.

[0067] In an embodiment of the present invention, illustratively, the server first obtains the contribution of the noise filtering cost parameter and the noise data estimation cost parameter respectively. The contribution can be understood as the importance and influence of each cost parameter in the model optimization process. For example, the server can determine the contribution of the two cost parameters through historical data analysis or expert evaluation. For example, after analysis, the contribution of the noise filtering cost parameter is determined to be 0.6, and the contribution of the noise data estimation cost parameter is determined to be 0.4. This means that in the model optimization process, the effect of noise filtering is more important. Next, the server performs multi-objective optimization on the two cost parameters according to their contribution. Multi-objective optimization is an optimization method that considers multiple objectives at the same time, aiming to find a balance so that all objectives can be relatively well satisfied. In this scenario, the server will use an optimization algorithm (such as genetic algorithm, particle swarm optimization, etc.) to consider the noise filtering effect and the accuracy of noise data estimation. Through continuous iteration and optimization, the server will eventually determine an overall cost parameter, which reflects the optimal trade-off between the two cost parameters under a given contribution. Finally, the server tunes the structural parameters of the model according to the overall cost parameters of the first noise elimination model. The structural parameters may include the number of layers of the model, the number of neurons, the selection of activation functions, etc. For example, if the overall cost parameters indicate that the noise filtering effect is more important, the server will increase the complexity of the model (such as increasing the number of layers or neurons) to improve the denoising ability of the model. Conversely, if the accuracy of noise data estimation is more important, the server will adjust the structure of the model to better capture and estimate the noise data. Through this series of tuning operations, the server will eventually determine a tower monitoring audio denoising model for performing noise filtering operations on the original tower monitoring audio. This model achieves a good balance between noise filtering and noise data estimation, and can more effectively remove the noise components in the tower monitoring audio.

[0068] In the embodiment of the present invention, the acquisition of the noise-free tower monitoring audio instance, the noisy tower monitoring audio instance matched by the noise-free tower monitoring audio instance, and the noise label of the noisy tower monitoring audio instance can be implemented through the following examples.

[0069] Collecting tower monitoring audio whose audio quality is greater than an audio quality threshold from a tower monitoring audio data pool as a noise-free tower monitoring audio instance; Degrading the sampling values ​​of the audio sample points in the noise-free iron tower monitoring audio instance to determine a processed noise-free iron tower monitoring audio instance; Performing signal enhancement processing on the processed noise-free iron tower monitoring audio instance to determine the pending noisy iron tower monitoring audio matched by the noise-free iron tower monitoring audio instance; Receiving a noise label for the monitoring audio of the pending noisy iron tower; According to the noise label of the pending noisy tower monitoring audio, the pending noisy tower monitoring audio and the noise-free tower monitoring audio instance, a noisy tower monitoring audio instance matching the noise-free tower monitoring audio instance is obtained; according to the noise label of the pending noisy tower monitoring audio, a noise label of the noisy tower monitoring audio instance is obtained.

[0070] In an embodiment of the present invention, exemplarily, the server screens the tower monitoring audio whose audio quality is higher than a set threshold from the tower monitoring audio data pool. These high-quality audios are regarded as noise-free tower monitoring audio instances. For example, the server sets an audio quality threshold of 0.9 (ranging from 0 to 1). Only when the comprehensive score of quality indicators such as audio clarity and signal-to-noise ratio is higher than 0.9, the audio will be selected as a noise-free tower monitoring audio instance. In order to simulate the noise conditions that may occur in the real environment, the server will degrade the collected noise-free tower monitoring audio instance. This usually involves adding different types of noise or distorting the audio signal to a certain extent. For example, the server will add Gaussian white noise to the sampling value of the audio sample point, or simulate the loss during signal transmission, so as to reduce the audio quality. After the degradation process, the server will perform signal enhancement processing on these processed audios, aiming to restore or enhance certain characteristics of the audio, but at the same time retain the added noise components. This operation is to simulate the process of enhancing noisy audio in reality. For example, the server will use an equalizer to adjust the audio spectrum, or use a noise reduction algorithm to try to reduce some of the noise, but at the same time ensure that the added noise is not completely removed. After processing the pending noisy tower monitoring audio, the server will receive noise labels for these audios from users or experts. These labels usually indicate the type and intensity of noise in the audio. For example, users will listen to the audio through an interface and score it or select predefined noise labels, such as "slight wind sound", "strong electromagnetic interference", etc. Finally, the server formally determines the noisy tower monitoring audio instances and their noise labels based on the received noise labels, the pending noisy tower monitoring audio, and the original noise-free tower monitoring audio instances. For example, if a pending audio is labeled as "slight wind sound", the server will store this audio and its corresponding noise-free version and noise label together as a dataset for subsequent model training and testing. Through the above steps, the server can build a dataset containing noise-free and noisy tower monitoring audio instances and corresponding noise labels, providing a rich data foundation for subsequent audio processing and analysis.

[0071] In an embodiment of the present invention, the noisy tower monitoring audio instance matching the noise-free tower monitoring audio instance is obtained based on the noise label of the pending noisy tower monitoring audio, the pending noisy tower monitoring audio and the noise-free tower monitoring audio instance, which can be implemented through the following examples.

[0072] According to the noise label, the bolt audio data in the pending noisy iron tower monitoring audio and the audio data corresponding to the bolt audio data in the noise-free iron tower monitoring audio instance are obtained; The audio data corresponding to the bolt audio data in the noise-free tower monitoring audio instance is used to replace the bolt audio data in the pending noisy tower monitoring audio, and the noisy tower monitoring audio instance that matches the noise-free tower monitoring audio instance is determined.

[0073] In an embodiment of the present invention, illustratively, the server first identifies and extracts the audio data related to the bolts from the pending noisy tower monitoring audio according to the noise tag. These bolt audio data are specific sounds of the tower bolts recorded in a noisy environment, and may include characteristic sounds such as bolt loosening and tightening. For example, if the noise tag contains "bolt loosening sound", the server will use audio processing technology, such as filtering, spectrum analysis, etc., to locate and extract the audio clips corresponding to these loosening sounds from the pending noisy tower monitoring audio. Next, the server will search for audio data corresponding to the extracted bolt audio data in the noiseless tower monitoring audio instance. This is because the bolt sound recorded in a noiseless environment can more clearly reflect the true state of the bolt, which is helpful for subsequent analysis and judgment. For example, the server can find a clear audio clip corresponding to the bolt loosening sound in the noiseless tower monitoring audio instance by comparing spectral features, time domain waveforms, etc. Finally, the server will use the clear audio data corresponding to the bolt audio data in the noiseless tower monitoring audio instance to replace the bolt audio data in the pending noisy tower monitoring audio. The purpose of this is to retain the noise environment in the original audio, while replacing the noise-interfered bolt sound with a clearer bolt sound, thereby obtaining a noisy tower monitoring audio instance that contains both the actual noise environment and the clear bolt sound. For example, if the original pending audio records the sound of loose bolts interfered by noise during time period T, the server will replace it with the clear loose bolt sound in the same time period in the noise-free audio. Through the above steps, the server is able to generate a noisy tower monitoring audio instance that matches the noise-free tower monitoring audio instance. This new audio instance not only retains the actual noise environment, but also contains clear bolt sound data, providing more accurate data support for subsequent tower status monitoring and fault diagnosis.

[0074] For illustrative purposes, the foregoing description is made with reference to specific embodiments. However, the above illustrative discussion is not intended to be exhaustive or to limit the present disclosure to the precise form disclosed. Numerous modifications and variations are possible in accordance with the above teachings. These embodiments are selected and described in order to best illustrate the principles of the present disclosure and its practical application, so that those skilled in the art can best utilize the present disclosure and utilize various embodiments with different modifications to suit the intended specific application.

Claims

1. A tower bolt abnormality monitoring method based on deep learning and voiceprint, characterized in that: The method comprises: Get the original tower monitoring audio; Calling a pre-trained tower monitoring audio denoising model to process the original tower monitoring audio to obtain a target tower monitoring audio; Obtaining the bolt soundprint to be analyzed of the target tower monitoring audio; The bolt voiceprint to be analyzed is loaded into a pre-trained bolt voiceprint anomaly classification model for voiceprint classification processing to obtain a target voiceprint anomaly classification result of the target tower monitoring audio.

2. The method according to claim 1, characterized in that: The bolt voiceprint anomaly classification model is obtained by the following methods, including: Obtaining a current bolt voiceprint instance of a target tower monitoring audio instance; the current bolt voiceprint instance is configured with a current voiceprint abnormality target value; According to the basic voiceprint classification model, an integrated voiceprint classification model and a target voiceprint classification model are constructed; the basic voiceprint classification model is obtained by loading the basic bolt voiceprint instance of the target tower monitoring audio instance into a preset classification model and performing bolt voiceprint abnormal classification training; the basic bolt voiceprint instance is configured with an initial target tower monitoring audio instance category label; the initial target tower monitoring audio instance category label is different from the current voiceprint abnormality target value; Loading the current bolt voiceprint instance into the integrated voiceprint classification model for voiceprint classification processing to obtain a first sample voiceprint type recognition result; Integrate the first sample voiceprint type recognition result and the current voiceprint abnormality target value to obtain a sample voiceprint abnormality target value; Loading the current bolt voiceprint instance into the target voiceprint classification model for voiceprint classification processing to obtain a second sample voiceprint type recognition result; According to the deviation between the second sample voiceprint type identification result and the sample voiceprint abnormal target value, a training process is performed on the target voiceprint classification model. The trained target voiceprint classification model is used to determine the bolt voiceprint abnormal classification model. The bolt voiceprint abnormal classification model is used to determine the type of the current voiceprint abnormal target value for the bolt voiceprint instance.

3. The method according to claim 2, characterized in that After obtaining the current bolt voiceprint instance of the target tower monitoring audio instance, the method further includes: Performing data cleaning processing on the current bolt voiceprint instance to obtain a transition bolt voiceprint instance, wherein the data cleaning processing is used to change the composition of the current bolt voiceprint instance; Loading the transition bolt voiceprint instance into the integrated voiceprint classification model for voiceprint classification processing to obtain a third sample voiceprint type recognition result; Loading the transition bolt voiceprint instance into the target voiceprint classification model for voiceprint classification processing to obtain a fourth sample voiceprint type recognition result; determining a first error parameter according to a deviation between the fourth sample voiceprint type recognition result and the third sample voiceprint type recognition result; According to the deviation between the second sample voiceprint type recognition result and the sample voiceprint abnormality target value, a training process is performed on the target voiceprint classification model, including: determining a second error parameter according to a deviation between the second sample voiceprint type recognition result and the sample voiceprint abnormality target value; Determining a target error parameter according to the first error parameter and the second error parameter; According to the target error parameter, the structural parameters of the target voiceprint classification model are adjusted.

4. The method according to claim 3, characterized in that The method further comprises: Adjusting the integrated voiceprint classification model according to the structural parameters of the target voiceprint classification model to obtain a candidate integrated voiceprint classification model; When the training cycle is the first preset cycle, after training for a training cycle of the second preset cycle, obtaining current structural parameters of the target voiceprint classification model; Processing the current structural parameters according to the state space model to obtain target structural parameters; Adjusting the candidate integrated voiceprint classification model according to the target structure parameter; When the preset training termination condition is reached, the candidate integrated voiceprint classification model is determined as the bolt voiceprint abnormality classification model, and the training cycle is the cycle of training the target voiceprint classification model using the current bolt voiceprint instance.

5. The method according to claim 4, characterized in that The step of processing the current structural parameters according to the state space model to obtain target structural parameters includes: Determine, according to the training cycle corresponding to the current structural parameter, a first influencing factor of the integrated voiceprint classification model and a second influencing factor of the target voiceprint classification model; the first influencing factor has a negative feedback relationship with the training cycle, and the second influencing factor has a positive feedback relationship with the training cycle; The current structural parameters are processed according to the first influencing factor, the second influencing factor and the state space model to obtain the target structural parameters.

6. The method according to claim 2, characterized in that The training method of the basic voiceprint classification model includes: Obtain the foundation bolt voiceprint instance of the target tower monitoring audio instance; Loading the basic bolt voiceprint instance into the preset classification model for voiceprint classification processing to obtain an initial sample voiceprint type recognition result; Determining a sample error parameter according to a deviation between the initial sample voiceprint type recognition result and the initial target tower monitoring audio instance category label; The structural parameters of the preset classification model are adjusted according to the sample error parameters, and the preset classification model that has completed training is determined as the basic voiceprint classification model.

7. The method according to claim 1, characterized in that The tower monitoring audio denoising model is obtained by: Collecting tower monitoring audio whose audio quality is greater than an audio quality threshold from a tower monitoring audio data pool as a noise-free tower monitoring audio instance; Degrading the sampling values ​​of the audio sample points in the noise-free iron tower monitoring audio instance to determine a processed noise-free iron tower monitoring audio instance; Performing signal enhancement processing on the processed noise-free iron tower monitoring audio instance to determine the pending noisy iron tower monitoring audio matched by the noise-free iron tower monitoring audio instance; Receiving a noise label for the monitoring audio of the pending noisy iron tower; According to the noise label, the bolt audio data in the pending noisy iron tower monitoring audio and the audio data corresponding to the bolt audio data in the noise-free iron tower monitoring audio instance are obtained; The audio data corresponding to the bolt audio data in the noise-free tower monitoring audio instance is used to replace the bolt audio data in the pending noisy tower monitoring audio, and the noisy tower monitoring audio instance that matches the noise-free tower monitoring audio instance is determined; and the noise label of the noisy tower monitoring audio instance is obtained according to the noise label of the pending noisy tower monitoring audio; the noise label indicates the noise data in the noisy tower monitoring audio instance; Using the first noise elimination model, performing a noise filtering operation on the noisy iron tower monitoring audio instance to determine the first iron tower monitoring filtered audio, performing noise data estimation on the noisy iron tower monitoring audio instance to determine the estimated noise data information of the noisy iron tower monitoring audio instance; According to the estimated noise data information, the bolt audio data in the noisy iron tower monitoring audio instance is obtained, and from the first iron tower monitoring filtered audio, the audio data having the same timing node as the bolt audio data in the noisy iron tower monitoring audio instance is collected, and the bolt audio data in the noisy iron tower monitoring audio instance is used to replace the audio data in the first iron tower monitoring filtered audio, so as to determine the iron tower monitoring filtered audio instance; Obtaining a noise filtering cost parameter of the first noise elimination model according to the tower monitoring filtered audio instance and the noise-free tower monitoring audio instance; Obtaining a noise data estimation cost parameter of the first noise elimination model according to the estimated noise data information and the noise label; Obtaining the contribution of the noise filtering cost parameter and the noise data estimation cost parameter respectively matching each other; According to the contribution degrees of the noise filtering cost parameter and the noise data estimation cost parameter respectively matched, multi-objective optimization processing is performed on the noise filtering cost parameter and the noise data estimation cost parameter to determine the overall cost parameter of the first noise elimination model; According to the overall cost parameter of the first noise elimination model, a tuning operation is performed on the structural parameters of the first noise elimination model to determine a tower monitoring audio denoising model for performing a noise filtering operation on the original tower monitoring audio.

8. The method according to claim 7, characterized in that The first noise elimination model includes a feature enhancement module, a noise filtering module and a noise data estimation module; the feature enhancement module includes a robust scaling submodule and a feature enhancement submodule; The noise filtering module includes a filtering coding submodule, a filtering feature mapping module and a filtering reverse robust scaling submodule; the noise data estimation module includes an estimation coding submodule, an estimation feature mapping module and an estimation reverse robust scaling submodule; Using the first noise elimination model, performing a noise filtering operation on the noisy iron tower monitoring audio instance, determining a first iron tower monitoring filtered audio, estimating noise data on the noisy iron tower monitoring audio instance, and determining estimated noise data information of the noisy iron tower monitoring audio instance, including: Using the robust scaling submodule, the noisy tower monitoring audio instance is robustly scaled according to the audio attribute matched by the noisy tower monitoring audio instance, and a robustly scaled noisy tower monitoring audio instance is determined; Using the feature enhancement submodule, feature enhancement is performed on the robustly scaled noisy iron tower monitoring audio instance to determine feature enhanced audio that matches the noisy iron tower monitoring audio instance; The output current intermediate feature audio of the current coding network is input into the subsequent coding network; the input of the initial coding network is the feature enhanced audio matched by the noisy tower monitoring audio instance; Using the post-order coding network, feature coding the output current intermediate feature audio to determine the initial feature coded audio matched by the post-order coding network; The initial feature coded audio matched by the post-order coding network and the output current intermediate feature audio are merged to determine the output intermediate feature audio of the post-order coding network; The output intermediate feature audio of the last encoding network is used as the first feature encoded audio of the noisy tower monitoring audio instance; Using the filtering feature mapping module, the first feature-coded audio is subjected to feature mapping processing to determine the first feature-mapped audio matched by the noisy tower monitoring audio instance; Using the filtering reverse robust scaling submodule, the first feature mapping audio is subjected to reverse robust scaling processing to determine the first tower monitoring filtered audio; Using the estimation coding submodule, feature coding is performed on the feature-enhanced audio matched by the noisy iron tower monitoring audio instance, and a second feature-coded audio matched by the noisy iron tower monitoring audio instance is determined; Using the estimated feature mapping module, the second feature-encoded audio is subjected to feature mapping processing to determine the second feature-mapped audio matched by the noisy tower monitoring audio instance; Using the estimated reverse robust scaling submodule, the second feature map audio is subjected to reverse robust scaling processing to determine the reverse robustly scaled second feature map audio; The second feature map audio after the reverse robust scaling is subjected to standardization processing to determine an estimated noise label of the noisy iron tower monitoring audio instance; the estimated noise label represents the noise data in the noisy iron tower monitoring audio instance; The estimated noise label of the noisy iron tower monitoring audio instance is regarded as the estimated noise data information of the noisy iron tower monitoring audio instance.

9. The method according to claim 7, characterized in that: The step of obtaining the bolt audio data in the noisy iron tower monitoring audio instance according to the estimated noise data information includes: According to the estimated noise data information, noise data in the noisy tower monitoring audio instance is obtained; The audio data other than the noise data in the noisy iron tower monitoring audio instance is regarded as the bolt audio data in the noisy iron tower monitoring audio instance.

10. A server system, characterized in that: The method comprises a server, wherein the server is used to execute the method described in any one of claims 1 to 9.

Images