Structural damage detection method based on voiceprint recognition

Abstract

The invention belongs to the technical field of damage detection, and particularly relates to a structural damage detection method based on voiceprint recognition, which comprises the following steps: step 1, a detection system is arranged and applied to a server, and the detection system is composed of a knock detection module, an audio anomaly detection module, a damage classification module, a damage rechecking module and a recognition interface module; 2, knocking a to-be-detected structure on site through a knocking detection module; step 3, segmenting all the audio files, and extracting key segments of the audio under the maximum knocking force from the audio files; and 4, training the generative adversarial network model based on optimal architecture search. According to the method, the generative adversarial network algorithm based on optimal architecture search is adopted for data expansion, the generative adversarial network architecture most suitable for voiceprint data characteristics can be automatically searched, the performance of the generative model is optimized, the learning and reproduction capabilities of voiceprint characteristics are improved, and recognition of structural damage is facilitated.

Description

Technical Field

[0001] The present invention relates to a damage detection method, and in particular to a structural damage detection method based on voiceprint recognition. Background Art

[0002] In engineering applications, non-destructive testing methods such as ultrasonic rebound, acoustic emission testing, x-ray projection, infrared scanning, etc. are often used to detect structural damage; the current non-destructive testing technology is far from meeting the requirements of actual engineering applications. At the same time, it is necessary to quickly judge the type, location, and degree of damage after collecting signals on site. The main reason is that the detection equipment is expensive, has different working principles and performance, especially for rapid detection of structures after disasters.

[0003] The survey was conducted based on the technical characteristics of 29 inspection solutions provided by structural inspection companies and research institutions. The statistical results showed that 21 of them used high-definition digital camera technology, 4 used laser scanning technology, and the other 4 used robot-assisted camera technology. They can generally accurately detect cracks with a width of 0.1-0.3 mm at a long distance of 5-10 meters.

[0004] However, this type of automatic rapid detection technology is mainly based on optical image detection technology, which can only detect damage on the surface of the structure. The accuracy and practicality of its non-destructive testing technology are obviously insufficient. In order to further understand the damage inside the structure, it is also necessary to develop an automated rapid detection method for the damage inside the structure to meet the needs of rapid response in emergency situations. Summary of the invention

[0005] The main purpose of the present disclosure is to provide a structural damage detection method based on voiceprint recognition to effectively solve the problems raised by the inventor in the above background technology.

[0006] To achieve the above object, the technical solution adopted by the present invention is as follows:

[0007] A structural damage detection method based on voiceprint recognition comprises the following steps:

[0008] Step 1: Layout the detection system and apply it to the server. The detection system consists of a knock detection module, an audio anomaly detection module, a damage classification module, a damage review module and an identification interface module;

[0009] Step 2: Use the knock detection module to knock on the structure to be detected on site, collect the voiceprint data set to obtain the audio file, and import the audio file into the audio anomaly detection module in turn, and then mark the risk type of the collected voiceprint data;

[0010] Step 3: Segment all audio files, extract key segments of the audio under maximum percussion force, mark the extracted key segments, and construct a training set;

[0011] Step 4: training a generative adversarial network model based on optimal architecture search. The trained generative adversarial network model based on optimal architecture search expands the key segments of the voiceprint data in step 3 and outputs the expanded voiceprint data. In training the generative adversarial network model based on optimal architecture search, the optimal architecture search strategy is adopted to define a search space S, which includes several generative adversarial network models with different configurations s.

[0012] Step 5: The audio file includes target audio data and corresponding expected execution values. The target audio data is sent to the recognition interface module, and the voiceprint features and user intentions returned by the recognition interface module are received. The voiceprint features and user intentions are generated by the recognition interface module calling the corresponding recognition algorithm and parsing the target audio data. The voiceprint features and user intentions are sent to the device execution interface to generate an execution command, and the device execution command is sent to the corresponding virtual device. The execution result returned by the virtual device according to the device execution command is received, and the voiceprint features and the execution results are compared with the expected execution values ​​respectively to generate a test report.

[0013] Step 6: Train a neural network model based on a multi-granularity honeypot optimization algorithm. The trained neural network model based on the multi-granularity honeypot optimization algorithm extracts the original feature representation of the expanded voiceprint data in step 5 and outputs the original feature representation; train an autoencoder based on latent Dirichlet allocation. The trained autoencoder based on latent Dirichlet allocation performs dimensionality reduction reconstruction on the original feature representation and outputs the reconstructed feature representation;

[0014] Step 7: According to the risk type marked in step 1, a classifier based on difference branch random forest is trained. The trained classifier based on difference branch random forest classifies the reconstructed feature representation in step 6, and determines the risk type of structural damage through classification;

[0015] Step 8: Build a deep learning network model and train it. Use the trained network model to detect sounds of unknown structures. If all segments are identified as normal after detection, it is considered that the detected structure has no defects; if any segment is identified as abnormal, it is considered that the detected structure has defects.

[0016] Preferably, in step one, the knock detection module is used to detect the structure to be detected and obtain a continuous sound signal, the audio anomaly detection module is used to receive the input of the continuous sound signal, analyze the voiceprint features of the sound signal groups in different time periods, and compare whether each voiceprint feature has changed. The voiceprint features analyzed by the audio anomaly detection module include wavelength, intensity, attenuation characteristics and sound wave spectrum, and when an abnormal value is detected, the feature vector composed of each voiceprint feature is output to the damage classification module. The damage classification module outputs the pending damage category of the internal damage through a machine learning classification algorithm, and the damage review module feeds back the pending damage category to the knock detection module.

[0017] Preferably, when an abnormal value is detected at a certain moment, the feature vector composed of various voiceprint features is used as input and imported into the damage classification module, and the pending damage category of the internal damage is output through the machine learning classification algorithm; the pending damage category is fed back to the knock detection module through the damage review module, and the knock detection module adjusts the frequency and intensity of the knock detection according to the damage voiceprint features pre-set by the damage review module, and then the feature vector composed of the pending damage category and the re-collected voiceprint features is input into the pre-trained classifier to verify whether the damage category is accurate, and the verified damage type is input into the identification interface module to record the damage type of the structure and the location of damage of different degrees.

[0018] Preferably, in step 4, the search goal is to find the optimal configuration S * , so that the value of the evaluation function of the adversarial generation network model is maximized, where the evaluation function is as follows:

[0019] F(s;θ g ,θ d )

[0020] Among them, s∈S, θ g and θ d These are the parameters of the adversarial generative network model.

[0021] Preferably, in step 4, training the adversarial generative network model based on optimal architecture search is specifically as follows:

[0022] Initialize the parameters of the generator G and discriminator D of the adversarial generation network model, and set the parameters of the generator G to θ g , the parameter of the discriminator D is θ d , the generator G learns the mapping function G(z, θ g ) maps the input noise vector z to the data space to generate simulated voiceprint data. The discriminator D uses the function D(x, θ d) evaluates the authenticity of the generated simulated voiceprint data, adopts the optimal architecture search strategy, defines the search space S, and the search space S contains several adversarial generative network models with different configurations s. The search goal is to find the optimal configuration s * , so that the evaluation function F(s;θ) of the adversarial generation network model g ,θ d ), where s∈S. In the searched adversarial generative network model, the generator loss function L is defined by dynamically adjusting the loss function to balance the capabilities of the generator G and the discriminator D. G (θ g ), the discriminator damage function L D (θ d ), the method of adjusting the training strategy by iteratively updating the loss function is expressed as:

[0023]

[0024]

[0025] Where E represents the expectation, logD(G(z, θ g );θ d ) represents the judgment of the discriminator D on the simulated voiceprint data generated by the generator G, x~p data (x) represents the probability distribution of x, D(·) represents the discriminator function, G(z, θ g ) represents the generator function, z~p z (z) represents the probability distribution that z obeys.

[0026] Preferably, in the step five, the target audio data is sent to the voiceprint recognition interface, and the voiceprint features and user intentions returned via the voiceprint recognition interface are received, including: when the measurement value of each target audio data exceeds the preset measurement value, each target audio data is segmented and processed in a preset order to obtain multiple audio data packets corresponding to each target audio data; the segmented multiple audio data packets are sent to the voiceprint recognition interface, and the multiple audio data packets received by the voiceprint recognition interface are processed based on the preset order to obtain the target audio data; when the voiceprint recognition interface detects the target audio data, the voiceprint recognition algorithm and semantic parsing algorithm corresponding to the voiceprint recognition interface are called; and the target audio data is parsed and processed using the voiceprint recognition algorithm and the semantic parsing algorithm to obtain the voiceprint features and user intentions corresponding to the target audio data.

[0027] In view of this, compared with the prior art, the beneficial effects of the present invention are:

[0028] (I) In this application, a generative adversarial network algorithm based on optimal architecture search is used for data expansion, which can automatically find the generative adversarial network architecture that best suits the characteristics of voiceprint data, optimize the performance of the generated model, improve the learning and reproduction capabilities of voiceprint features, and facilitate the identification of structural damage.

[0029] (ii) In this application, an autoencoder structure based on latent Dirichlet allocation is adopted, so that the encoding process can take into account the potential topic distribution of the data, improve the model's ability to understand the data generation process, and propose an improved random forest classification algorithm, which not only improves the classification accuracy by increasing the differences between trees and applying a sparsity improvement mechanism.

[0030] (III) In the present application, a recurrent neural network model capable of processing variable-length sequence input is adopted, a recurrent neural network capable of accepting variable-length sequence as input is used as the input layer, and a deep neural network model is constructed on this basis, thereby successfully avoiding the drawbacks of traditional use of convolutional neural networks for audio processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 The figure is a flow chart of the structural damage detection method based on voiceprint recognition provided by the present invention;

[0032] Figure 2 Shown is a schematic diagram of the structure of the detection system. DETAILED DESCRIPTION

[0033] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0034] See also Figure 1-2 , the present invention provides the following embodiments:

[0035] A structural damage detection method based on voiceprint recognition comprises the following steps:

[0036] Step 1: Layout the detection system and apply it to the server. The detection system consists of a knock detection module, an audio anomaly detection module, a damage classification module, a damage review module, and an identification interface module;

[0037] Step 2: Use the knock detection module to knock on the structure to be detected on site, collect the voiceprint data set to obtain the audio file, and import the audio file into the audio anomaly detection module in turn, and then mark the risk type of the collected voiceprint data;

[0038] Step 3: Segment all audio files, extract key segments of the audio under maximum percussion force, mark the extracted key segments, and construct a training set;

[0039] Step 4: training a generative adversarial network model based on optimal architecture search. The trained generative adversarial network model based on optimal architecture search expands the key segments of the voiceprint data in step 3 and outputs the expanded voiceprint data. In training the generative adversarial network model based on optimal architecture search, the optimal architecture search strategy is adopted to define a search space S, which includes several generative adversarial network models with different configurations s.

[0040] Step 5: The audio file includes the target audio data and the corresponding expected execution value. The target audio data is sent to the recognition interface module, and the voiceprint features and user intentions returned by the recognition interface module are received. The voiceprint features and user intentions are used by the recognition interface module to call the corresponding recognition algorithm, and the target audio data is parsed and generated. The voiceprint features and user intentions are sent to the device execution interface to generate an execution command, and the device execution command is sent to the corresponding virtual device. The execution result returned by the virtual device according to the device execution command is received, and the voiceprint features and execution results are compared with the expected execution values ​​respectively to generate a test report;

[0041] Step 6: Train a neural network model based on a multi-granularity honeypot optimization algorithm. The trained neural network model based on the multi-granularity honeypot optimization algorithm extracts the original feature representation of the expanded voiceprint data in step 5 and outputs the original feature representation; train an autoencoder based on latent Dirichlet allocation. The trained autoencoder based on latent Dirichlet allocation performs dimensionality reduction reconstruction on the original feature representation and outputs the reconstructed feature representation;

[0042] Step 7: According to the risk type marked in step 1, a classifier based on difference branch random forest is trained. The trained classifier based on difference branch random forest classifies the reconstructed feature representation in step 6, and determines the risk type of structural damage through classification;

[0043] Step 8: Build a deep learning network model and train it. Use the trained network model to detect sounds of unknown structures. If all segments are identified as normal after detection, it is considered that the detected structure has no defects; if any segment is identified as abnormal, it is considered that the detected structure has defects.

[0044] Specifically, in step one, the knock detection module is used to detect the structure to be detected and obtain a continuous sound signal, the audio anomaly detection module is used to receive the input of the continuous sound signal, analyze the voiceprint features of the sound signal groups in different time periods, and compare whether the various voiceprint features have changed. The voiceprint features analyzed by the audio anomaly detection module include wavelength, intensity, attenuation characteristics and sound wave spectrum, and when an abnormal value is detected, the feature vector composed of various voiceprint features is output to the damage classification module. The damage classification module outputs the pending damage category of the internal damage through a machine learning classification algorithm, and the damage review module feeds back the pending damage category to the knock detection module.

[0045] Specifically, when an abnormal value is detected at a certain moment, the feature vector composed of various voiceprint features is used as input and imported into the damage classification module, and the pending damage category of the internal damage is output through the machine learning classification algorithm; the pending damage category is fed back to the knock detection module through the damage review module, and according to the damage voiceprint features pre-set by the damage review module, the knock detection module adjusts the frequency and intensity of the knock detection and performs the knock detection again, and then inputs the feature vector composed of the pending damage category and the re-collected voiceprint features into the pre-trained classifier to verify whether the damage category is accurate, and inputs the verified damage type into the recognition interface module to record the damage type of the structure and the location of damage of different degrees.

[0046] Specifically, in step 4, the search goal is to find the optimal configuration s * , so that the value of the evaluation function of the adversarial generation network model is maximized, where the evaluation function is as follows:

[0047] F(s;θ g ,θ d )

[0048] Among them, s∈S, θ g and θ d These are the parameters of the adversarial generative network model.

[0049] Specifically, in step 4, the training of the adversarial generative network model based on optimal architecture search is as follows:

[0050] Initialize the parameters of the generator G and discriminator D of the adversarial generation network model, and set the parameters of the generator G to θ g , the parameter of the discriminator D is θ d , the generator G learns the mapping function G(z, θ g ) maps the input noise vector z to the data space to generate simulated voiceprint data. The discriminator D uses the function D(x, θ d) evaluates the authenticity of the generated simulated voiceprint data, adopts the optimal architecture search strategy, defines the search space S, and the search space S contains several adversarial generative network models with different configurations s. The search goal is to find the optimal configuration s * , so that the evaluation function F(s;θ) of the adversarial generation network model g ,θ d ), where s∈S. In the searched adversarial generative network model, the generator loss function L is defined by dynamically adjusting the loss function to balance the capabilities of the generator G and the discriminator D. G (θ g ), the discriminator damage function L D (θ d ), the method of adjusting the training strategy by iteratively updating the loss function is expressed as:

[0051]

[0052]

[0053] Where E represents the expectation, logD(G(z, θ g );θ d ) represents the judgment of the discriminator D on the simulated voiceprint data generated by the generator G, x~p data (x) represents the probability distribution of x, D(·) represents the discriminator function, G(z, θ g ) represents the generator function, z~p z (z) represents the probability distribution that z obeys.

[0054] Specifically, in step five, the target audio data is sent to the voiceprint recognition interface, and the voiceprint features and user intentions returned via the voiceprint recognition interface are received, including: when the measurement value of each target audio data exceeds the preset measurement value, each target audio data is segmented and processed in a preset order to obtain multiple audio data packets corresponding to each target audio data; the segmented multiple audio data packets are sent to the voiceprint recognition interface, and the multiple audio data packets received by the voiceprint recognition interface are processed based on the preset order to obtain the target audio data; when the voiceprint recognition interface detects the target audio data, the voiceprint recognition algorithm and semantic parsing algorithm corresponding to the voiceprint recognition interface are called; the target audio data is parsed and processed using the voiceprint recognition algorithm and the semantic parsing algorithm to obtain the voiceprint features and user intentions corresponding to the target audio data.

[0055] The specific implementation methods of this embodiment are as follows: an autoencoder structure based on latent Dirichlet allocation is adopted, so that the encoding process can take into account the potential topic distribution of the data, improve the model's ability to understand the data generation process, and propose an improved random forest classification algorithm. By increasing the differences between trees and applying a sparsity improvement mechanism, not only the classification accuracy is improved; a generative adversarial network algorithm based on optimal architecture search is adopted for data expansion, which can automatically find the generative adversarial network architecture that best suits the characteristics of voiceprint data, optimize the performance of the generated model, improve the learning and reproduction capabilities of voiceprint features, and facilitate the identification of structural damage; a recurrent neural network model that can process variable-length sequence inputs is adopted, a recurrent neural network that can accept variable-length sequences as inputs is used as the input layer, and a deep neural network model is constructed on this basis, thereby successfully avoiding the drawbacks of traditional use of convolutional neural networks to process audio.

[0056] In the description of this specification, the description with reference to the terms "one embodiment", "example", "specific example", etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0057] The preferred embodiments of the present invention disclosed above are only used to help illustrate the present invention. The preferred embodiments do not describe all the details in detail, nor do they limit the invention to the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present invention, so that those skilled in the art can understand and use the present invention well. The present invention is limited only by the claims and their full scope and equivalents.

Claims

1. A structural damage detection method based on voiceprint recognition, characterized in that: The following steps are involved: Step 1: Layout the detection system and apply it to the server. The detection system consists of a knock detection module, an audio anomaly detection module, a damage classification module, a damage review module and an identification interface module; Step 2: Use the knock detection module to knock on the structure to be detected on site, collect the voiceprint data set to obtain the audio file, and import the audio file into the audio anomaly detection module in turn, and then mark the risk type of the collected voiceprint data; Step 3: Segment all audio files, extract key segments of the audio under maximum percussion force, mark the extracted key segments, and construct a training set; Step 4: training a generative adversarial network model based on optimal architecture search. The trained generative adversarial network model based on optimal architecture search expands the key segments of the voiceprint data in step 3 and outputs the expanded voiceprint data. In training the generative adversarial network model based on optimal architecture search, the optimal architecture search strategy is adopted to define a search space S, which includes several generative adversarial network models with different configurations s. Step 5: The audio file includes target audio data and corresponding expected execution values. The target audio data is sent to the recognition interface module, and the voiceprint features and user intentions returned by the recognition interface module are received. The voiceprint features and user intentions are generated by the recognition interface module calling the corresponding recognition algorithm and parsing the target audio data. The voiceprint features and user intentions are sent to the device execution interface to generate an execution command, and the device execution command is sent to the corresponding virtual device. The execution result returned by the virtual device according to the device execution command is received, and the voiceprint features and the execution results are compared with the expected execution values ​​respectively to generate a test report. Step 6: training a neural network model based on a multi-granularity honeypot optimization algorithm, extracting the original feature representation of the expanded voiceprint data in step 5 from the trained neural network model based on a multi-granularity honeypot optimization algorithm, and outputting the original feature representation; Training an autoencoder based on latent Dirichlet allocation, performing dimensionality reduction reconstruction on the original feature representation by the trained autoencoder based on latent Dirichlet allocation, and outputting the reconstructed feature representation; Step 7: According to the risk type marked in step 1, a classifier based on difference branch random forest is trained. The trained classifier based on difference branch random forest classifies the reconstructed feature representation in step 6, and determines the risk type of structural damage through classification; Step 8: Build a deep learning network model and train it. Use the trained network model to detect sounds of unknown structures. If all segments are identified as normal after detection, it is considered that the detected structure has no defects; if any segment is identified as abnormal, it is considered that the detected structure has defects.

2. The structural damage detection method based on voiceprint recognition according to claim 1 is characterized in that: In the step one, the knock detection module is used to detect the structure to be detected and obtain a continuous sound signal, the audio anomaly detection module is used to receive the input of the continuous sound signal, analyze the voiceprint features of the sound signal groups in different time periods, and compare whether the various voiceprint features have changed. The voiceprint features analyzed by the audio anomaly detection module include wavelength, intensity, attenuation characteristics and sound wave spectrum, and when an abnormal value is detected, the feature vector composed of the various voiceprint features is output to the damage classification module. The damage classification module outputs the pending damage category of the internal damage through a machine learning classification algorithm, and the damage review module feeds back the pending damage category to the knock detection module.

3. The structural damage detection method based on voiceprint recognition according to claim 1 is characterized in that: In step 4, the search goal is to find the optimal configuration S * , so that the value of the evaluation function of the adversarial generation network model is maximized, where the evaluation function is as follows: F(s;θ g ,the d ) Among them, s∈S, θ g and θ d These are the parameters of the adversarial generative network model.

4. The structural damage detection method based on voiceprint recognition according to claim 2 is characterized in that: When an abnormal value is detected at a certain moment, the feature vector composed of various voiceprint features is used as input and imported into the damage classification module, and the pending damage category of the internal damage is output through the machine learning classification algorithm; the pending damage category is fed back to the knock detection module through the damage review module, and according to the damage voiceprint features pre-set by the damage review module, the knock detection module adjusts the frequency and intensity of the knock detection and performs the knock detection again, and then inputs the feature vector composed of the pending damage category and the re-collected voiceprint features into the pre-trained classifier to verify whether the damage category is accurate, and inputs the verified damage type into the recognition interface module to record the damage type of the structure and the location of damage of different degrees.

5. The structural damage detection method based on voiceprint recognition according to claim 1 is characterized in that: In the step five, the target audio data is sent to the voiceprint recognition interface, and the voiceprint features and user intentions returned by the voiceprint recognition interface are received, including: when the measurement value of each target audio data exceeds the preset measurement value, each target audio data is segmented and processed in a preset order to obtain a plurality of audio data packets corresponding to each target audio data; the segmented plurality of audio data packets are sent to the voiceprint recognition interface, and the plurality of audio data packets received by the voiceprint recognition interface are processed based on the preset order to obtain the target audio data; when the voiceprint recognition interface detects the target audio data, the voiceprint recognition algorithm and the semantic parsing algorithm corresponding to the voiceprint recognition interface are called; the target audio data is parsed and processed using the voiceprint recognition algorithm and the semantic parsing algorithm to obtain the voiceprint features and user intentions corresponding to the target audio data.

6. The structural damage detection method based on voiceprint recognition according to claim 3 is characterized by: In the step 4, training the adversarial generative network model based on optimal architecture search is specifically as follows: Initialize the parameters of the generator G and discriminator D of the adversarial generation network model, and set the parameters of the generator G to θ g , the parameter of the discriminator D is θ d , the generator G learns the mapping function G(z, θ g ) maps the input noise vector z to the data space to generate simulated voiceprint data. The discriminator D uses the function D(x, θ d ) evaluates the authenticity of the generated simulated voiceprint data, adopts the optimal architecture search strategy, defines the search space S, and the search space S contains several adversarial generative network models with different configurations s. The search goal is to find the optimal configuration S * , so that the evaluation function F(s;θ) of the adversarial generation network model g ,θ d ), where s∈S. In the searched adversarial generative network model, the generator loss function L is defined by dynamically adjusting the loss function to balance the capabilities of the generator G and the discriminator D. G (θ g ), the discriminator damage function L D (θ d ), the method of adjusting the training strategy by iteratively updating the loss function is expressed as: Where E represents the expectation, logD(G(z, θ g );θ d ) represents the judgment of the discriminator D on the simulated voiceprint data generated by the generator G, x~p data (x) represents the probability distribution of x, D(·) represents the discriminator function, G(z, θ g ) represents the generator function, z~p z (z) represents the probability distribution that z obeys.

Images