Intelligent action recognition point detection hammer based on acoustic signal analysis

Abstract

The application provides an intelligent action recognition point inspection hammer based on acoustic signal analysis, and belongs to the technical field of intelligent recognition, and specifically comprises: a microphone array, a processing circuit, and a display screen; wherein the microphone array is responsible for acquiring acoustic signals; the processing circuit is responsible for judging the action type, the fault type and the fault probability of the object to be detected; and the display screen is responsible for outputting the alarm signal, the fault type and the fault probability, so as to ensure the efficiency and reliability of fault maintenance.

Description

Technical Field

[0001] This invention belongs to the field of intelligent recognition technology, and in particular relates to an intelligent motion recognition inspection hammer based on acoustic signal analysis. Background Technology

[0002] To enable regular inspections of equipment such as railway tracks, inspection hammers collect acoustic signals from the objects under inspection, allowing for easy identification of internal faults such as cracks or air raids. During inspection, actions such as hooking or pulling may also be used to inspect the objects. For different operations, the acoustic characteristic signals obtained may reflect different fault characteristics even for the same feature quantity. Therefore, how to identify the actions of different inspection hammers and extract and classify the corresponding fault characteristics based on the identification results has become a pressing technical problem.

[0003] To address the aforementioned technical problems, this invention provides an intelligent motion recognition inspection hammer based on acoustic signal analysis. Summary of the Invention

[0004] To achieve the objectives of this invention, the following technical solution is adopted: According to one aspect of the present invention, an intelligent action recognition method based on acoustic signal analysis is provided.

[0005] An intelligent action recognition method based on acoustic signal analysis, characterized in that it specifically includes: S11 acquires acoustic signals in real time through the inspection hammer and determines whether there are characteristic acoustic signals within a preset period. If yes, proceed to step S12; otherwise, output an alarm signal. S12 segments the characteristic acoustic signal based on a preset acoustic amplitude and a preset time to obtain an acoustic feature sequence, extracts motion acoustic features based on the acoustic feature sequence, and identifies the motion of the inspection hammer based on the motion acoustic features to obtain the motion type. S13 determines the acoustic feature extraction method based on the action type, extracts the acoustic feature sequence through the acoustic feature extraction method to obtain diagnostic acoustic features, and performs fault diagnosis on the object to be detected based on the diagnostic acoustic features to obtain the fault type and fault probability of the initial action, and determines whether the fault probability is greater than the probability threshold. If yes, proceed to step S14; otherwise, output the fault type and fault probability of the object to be detected based on the fault type and fault probability of the initial action. S14 outputs recommended action types and operation counts based on the fault probability, and determines the fault type and fault probability of the confirmation action by combining the acoustic sequence features generated by the recommended action type. It also determines the fault type and fault probability of the object to be detected by combining the fault type and fault probability of the confirmation action and the fault type and fault probability of the initial action.

[0006] By judging whether characteristic acoustic signals exist within a preset period, the technical problem of increased power consumption caused by forgetting to turn off the inspection hammer is solved, making the inspection hammer more durable and ensuring its reliability and stability.

[0007] The action type of the inspection hammer is identified by the acoustic characteristics of the action, thereby realizing the judgment of the action type of the inspection hammer from the perspective of acoustic signals. This also lays the foundation for further improving the accuracy and reliability of the inspection hammer in diagnosing fault types.

[0008] By determining the acoustic feature extraction method based on the action type, acoustic features can be determined based on different action types. While ensuring the efficiency of the evaluation, the accuracy of the evaluation is further improved. Furthermore, based on the diagnostic acoustic features, fault diagnosis is performed on the object to be detected to obtain the fault type and fault probability of the initial action. This enables the judgment of the fault type from the perspective of acoustic features, thereby improving the reliability of the judgment.

[0009] By combining the fault type and probability of the confirmation action with the fault type and probability of the initial action, the fault type and probability of the object to be detected are determined, thereby enabling the judgment of the fault type from multiple perspectives, further improving the accuracy of the judgment. Furthermore, by setting the fault probability, more detailed references are provided for maintenance personnel, further improving the efficiency of maintenance.

[0010] A further technical solution is that the characteristic acoustic signal is determined based on the preset acoustic frequency characteristics and the preset acoustic amplitude characteristics, and when the acoustic signal within the preset period does not include the characteristic acoustic signal, it is determined that the inspection hammer has been forgotten to be turned off, and an alarm signal is output.

[0011] A further technical solution is that, before extracting the acoustic feature sequence, the feature acoustic signal needs to be filtered and denoised, specifically using a Gaussian filtering method.

[0012] A further technical solution involves segmenting the characteristic acoustic signal to obtain an acoustic feature sequence, specifically including: Based on the preset acoustic amplitude, the characteristic acoustic signal is located to obtain multiple candidate acoustic localization points; Based on the candidate acoustic positioning point, the characteristic acoustic signal within a preset time period centered on the candidate acoustic positioning point is obtained as the acoustic feature sequence.

[0013] A further technical solution is that the specific steps for recognizing the action type are as follows: S21 extracts MFCC features based on the acoustic feature sequence, and obtains the similarity between the acoustic feature sequence and different action types as MFCC similarity based on the MFCC features and the historical MFCC features of different action types. S22 extracts LPCC features based on the acoustic feature sequence, and obtains the similarity between the acoustic feature sequence and different action types as LPCC similarity based on the LPCC features and the historical LPCC features of different action types. S23 determines whether the action type can be determined based on the MFCC similarity and the LPCC similarity. If yes, the action type is determined based on the MFCC similarity and the LPCC similarity. If no, proceed to step S24. S24 determines the comprehensive similarity based on the energy spectrum of the acoustic feature sequence, the MFCC similarity, and the LPCC similarity, and determines the action type based on the comprehensive similarity.

[0014] A further technical solution is that the MFCC similarity is determined based on the Mahalanobis distance between the MFCC features and the historical MFCC features of different action types.

[0015] A further technical solution is that, when the action type is a hooking type, the acoustic feature extraction method includes short-time frame energy ratio, intercepted energy pulse width, power spectral density, and mean square frequency; when the action type is a striking type, the acoustic feature extraction method includes waveform factor, short-time energy, spectral energy concentration, and center of gravity frequency.

[0016] A further technical solution involves determining the fault type and fault probability of the initial action when the action type is a hooking type. S31 obtains the short-time frame energy ratio and the truncated energy pulse width based on the diagnostic acoustic features, and obtains the fused acoustic temporal features based on the short-time frame energy ratio and the truncated energy pulse width; S32 obtains the power spectral density and mean square frequency based on the diagnostic acoustic features, and obtains the fused acoustic frequency domain features based on the power spectral density and mean square frequency. S33 uses the fused acoustic time-domain features and the fused acoustic frequency-domain features to obtain the fault type of the initial action by adopting a classification model based on the SVM algorithm. S34 constructs a prediction model based on the fault type, the fused acoustic time-domain features, and the fused acoustic frequency-domain features to obtain the fault probability of the fault type.

[0017] A further technical solution involves determining the fault type and fault probability of the object to be detected through the following specific steps: S41 determines whether the fault type of the confirmed action is consistent with the fault type of the initial action. If yes, proceed to step S42; otherwise, output that the fault type cannot be determined temporarily and further testing is required. S42 determines the weight of the failure probability of the confirmed action based on the number of actions of the recommended action type and the recommended action type; determines the weight of the failure probability of the initial action based on the action type of the initial action; S43 determines the failure probability of the object to be detected based on the weight of the failure probability of the initial action and the weight of the failure probability of the confirmation action, as well as the failure probability of the initial action and the failure probability of the confirmation action, and outputs an alarm signal when the failure probability of the object to be detected is greater than a preset value.

[0018] On the other hand, the present invention provides an intelligent action recognition point inspection hammer based on acoustic signal analysis, employing the aforementioned intelligent action recognition method based on acoustic signal analysis, specifically including: Microphone array, processing circuitry, display screen; The microphone array is responsible for acquiring acoustic signals; The processing circuit is responsible for determining the action type, as well as the fault type and fault probability of the object to be detected. The display screen is responsible for outputting alarm signals, fault types, and fault probabilities.

[0019] Other features and advantages will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention are realized and obtained through the structures particularly pointed out in the description and the drawings.

[0020] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0021] The above and other features and advantages of the present invention will become more apparent from a detailed description of exemplary embodiments thereof with reference to the accompanying drawings.

[0022] Figure 1 This is a flowchart of an intelligent action recognition method based on acoustic signal analysis according to Embodiment 1.

[0023] Figure 2 This is a block diagram of an intelligent motion recognition inspection hammer based on acoustic signal analysis according to Embodiment 2. Detailed Implementation

[0024] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the embodiments set forth herein; rather, they are provided so that the invention will be thorough and complete, and the concept of the exemplary embodiments will be fully conveyed to those skilled in the art. The same reference numerals in the drawings denote the same or similar structures, and therefore their detailed description will be omitted.

[0025] The terms “a,” “one,” “the,” and “the” are used to indicate the existence of one or more elements / components / etc.; the terms “including” and “having” are used to indicate an open-ended meaning of inclusion and that other elements / components / etc. may exist in addition to the listed elements / components / etc.

[0026] Example 1 To solve the above problems, according to one aspect of the present invention, such as Figure 1 As shown, an intelligent action recognition method based on acoustic signal analysis is provided, characterized by specifically including: S11 acquires acoustic signals in real time through the inspection hammer and determines whether there are characteristic acoustic signals within a preset period. If yes, proceed to step S12; otherwise, output an alarm signal. It should be noted that the characteristic acoustic signal is determined based on the preset acoustic frequency characteristics and preset acoustic amplitude characteristics, and when the acoustic signal within the preset period does not include the characteristic acoustic signal, it is determined that the inspection hammer has been forgotten to be turned off, and an alarm signal is output.

[0027] In this embodiment, by judging whether there is a characteristic acoustic signal in the acoustic signal within a preset period, the technical problem of increased power consumption caused by forgetting to turn off the point inspection hammer is solved, making the point inspection hammer more durable and ensuring its reliability and stability.

[0028] S12 segments the characteristic acoustic signal based on a preset acoustic amplitude and a preset time to obtain an acoustic feature sequence, extracts motion acoustic features based on the acoustic feature sequence, and identifies the motion of the inspection hammer based on the motion acoustic features to obtain the motion type. It should be noted that before extracting the acoustic feature sequence, the feature acoustic signal needs to be filtered and denoised, specifically using a Gaussian filtering method.

[0029] It should be noted that the acoustic feature sequence is obtained by segmenting the characteristic acoustic signal, specifically including: Based on the preset acoustic amplitude, the characteristic acoustic signal is located to obtain multiple candidate acoustic localization points; Based on the candidate acoustic positioning point, the characteristic acoustic signal within a preset time period centered on the candidate acoustic positioning point is obtained as the acoustic feature sequence.

[0030] It should be noted that the specific steps for identifying the action type are as follows: S21 extracts MFCC features based on the acoustic feature sequence, and obtains the similarity between the acoustic feature sequence and different action types as MFCC similarity based on the MFCC features and the historical MFCC features of different action types. S22 extracts LPCC features based on the acoustic feature sequence, and obtains the similarity between the acoustic feature sequence and different action types as LPCC similarity based on the LPCC features and the historical LPCC features of different action types. S23 determines whether the action type can be determined based on the MFCC similarity and the LPCC similarity. If yes, the action type is determined based on the MFCC similarity and the LPCC similarity. If no, proceed to step S24. S24 determines the comprehensive similarity based on the energy spectrum of the acoustic feature sequence, the MFCC similarity, and the LPCC similarity, and determines the action type based on the comprehensive similarity.

[0031] In this example, the action type of the inspection hammer is identified by the acoustic features of the action, thereby realizing the judgment of the action type of the inspection hammer from the perspective of acoustic signals. This also lays the foundation for further improving the accuracy and reliability of the inspection hammer in diagnosing fault types.

[0032] S13 determines the acoustic feature extraction method based on the action type, extracts the acoustic feature sequence through the acoustic feature extraction method to obtain diagnostic acoustic features, and performs fault diagnosis on the object to be detected based on the diagnostic acoustic features to obtain the fault type and fault probability of the initial action, and determines whether the fault probability is greater than the probability threshold. If yes, proceed to step S14; otherwise, output the fault type and fault probability of the object to be detected based on the fault type and fault probability of the initial action. It should be noted that the MFCC similarity is determined based on the Mahalanobis distance between the MFCC features and the historical MFCC features of different action types.

[0033] It should be noted that when the action type is hooking, the acoustic feature extraction method includes short-time frame energy ratio, truncated energy pulse width, power spectral density, and mean square frequency. When the action type is striking, the acoustic feature extraction method includes waveform factor, short-time energy, spectral energy concentration, and center of gravity frequency.

[0034] It should be noted that when the action type is a hook-pull type, the specific steps for determining the fault type and fault probability of the initial action are as follows: S31 obtains the short-time frame energy ratio and the truncated energy pulse width based on the diagnostic acoustic features, and obtains the fused acoustic temporal features based on the short-time frame energy ratio and the truncated energy pulse width; S32 obtains the power spectral density and mean square frequency based on the diagnostic acoustic features, and obtains the fused acoustic frequency domain features based on the power spectral density and mean square frequency. S33 uses the fused acoustic time-domain features and the fused acoustic frequency-domain features to obtain the fault type of the initial action by adopting a classification model based on the SVM algorithm. S34 constructs a prediction model based on the fault type, the fused acoustic time-domain features, and the fused acoustic frequency-domain features to obtain the fault probability of the fault type.

[0035] In this embodiment, by determining the acoustic feature extraction method based on the action type, the acoustic features can be determined based on different action types. While ensuring the efficiency of the evaluation, the accuracy of the evaluation is further improved. Furthermore, based on the diagnostic acoustic features, the fault diagnosis of the object to be detected is performed to obtain the fault type and fault probability of the initial action, thereby realizing the judgment of the fault type from the perspective of acoustic features and improving the reliability of the judgment.

[0036] S14 outputs recommended action types and operation counts based on the fault probability, and determines the fault type and fault probability of the confirmation action by combining the acoustic sequence features generated by the recommended action type. It also determines the fault type and fault probability of the object to be detected by combining the fault type and fault probability of the confirmation action and the fault type and fault probability of the initial action.

[0037] It should be noted that the specific steps for determining the fault type and fault probability of the object to be detected are as follows: S41 determines whether the fault type of the confirmed action is consistent with the fault type of the initial action. If yes, proceed to step S42; otherwise, output that the fault type cannot be determined temporarily and further testing is required. S42 determines the weight of the failure probability of the confirmed action based on the number of actions of the recommended action type and the recommended action type; determines the weight of the failure probability of the initial action based on the action type of the initial action; S43 determines the failure probability of the object to be detected based on the weight of the failure probability of the initial action and the weight of the failure probability of the confirmation action, as well as the failure probability of the initial action and the failure probability of the confirmation action, and outputs an alarm signal when the failure probability of the object to be detected is greater than a preset value.

[0038] In this embodiment, by combining the fault type and fault probability of the confirmation action and the fault type and fault probability of the initial action, the fault type and fault probability of the object to be detected are determined, thereby realizing the judgment of the fault type from multiple perspectives, further improving the accuracy of the judgment. Furthermore, by setting the fault probability, more detailed references are provided for maintenance personnel, further improving the efficiency of maintenance.

[0039] Example 2 like Figure 2 As shown, this invention provides an intelligent action recognition point inspection hammer based on acoustic signal analysis. The method employing the aforementioned intelligent action recognition method based on acoustic signal analysis specifically includes: Microphone array, processing circuitry, display screen; The microphone array is responsible for acquiring acoustic signals; The processing circuit is responsible for determining the action type, as well as the fault type and fault probability of the object to be detected. The display screen is responsible for outputting alarm signals, fault types, and fault probabilities.

[0040] In the several embodiments provided in this application, it should be understood that the disclosed systems and methods can also be implemented in other ways. The system embodiments described above are merely illustrative; for example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0041] In addition, the functional modules in the various embodiments of the present invention can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0042] If the functionality is implemented as a software module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0043] Based on the above-described preferred embodiments of the present invention, and through the foregoing description, those skilled in the art can make various changes and modifications without departing from the inventive concept. The technical scope of this invention is not limited to the contents of the specification, but must be determined according to the scope of the claims.

Claims

1. An intelligent action recognition method based on acoustic signal analysis, characterized in that, Specifically, it includes: Acoustic signals are acquired in real time by a point inspection hammer, and it is determined whether characteristic acoustic signals exist within a preset period. If so, the process proceeds to the next step; otherwise, an alarm signal is output. The acoustic feature sequence is obtained by segmenting the feature acoustic signal based on a preset acoustic amplitude and a preset time. Specifically, this includes: locating the feature acoustic signal based on the preset acoustic amplitude to obtain multiple candidate acoustic location points. Based on the candidate acoustic positioning point, characteristic acoustic signals within a preset time period centered on the candidate acoustic positioning point are obtained as an acoustic feature sequence; and action acoustic features are extracted based on the acoustic feature sequence, and the action type of the point inspection hammer is identified based on the action acoustic features. The acoustic feature extraction method is determined based on the action type. When the action type is hooking, the acoustic feature extraction method includes short-time frame energy ratio, truncated energy pulse width, power spectral density, and mean square frequency. When the action type is striking, the acoustic feature extraction method includes waveform factor, short-time energy, spectral energy concentration, and center of gravity frequency. Diagnostic acoustic features are obtained by extracting the acoustic feature sequence through the acoustic feature extraction method. Based on the diagnostic acoustic features, fault diagnosis is performed on the object to be detected to obtain the fault type and fault probability of the initial action. It is then determined whether the fault probability is greater than the probability threshold. If so, the next step is initiated. If not, the fault type and fault probability of the object to be detected are output based on the fault type and fault probability of the initial action. Based on the fault probability, a recommended action type and number of operations are output. The fault type and fault probability of the confirmation action are determined by combining the acoustic feature sequence generated by the recommended action type. The fault type and fault probability of the object to be detected are determined by combining the fault type and fault probability of the confirmation action and the fault type and fault probability of the initial action.

2. The intelligent action recognition method as described in claim 1, characterized in that, The characteristic acoustic signal is determined based on preset acoustic frequency characteristics and preset acoustic amplitude characteristics. When the acoustic signal within the preset period does not include the characteristic acoustic signal, it is determined that the inspection hammer has been forgotten to be turned off, and an alarm signal is output.

3. The intelligent action recognition method as described in claim 1, characterized in that, Before extracting the acoustic feature sequence, the feature acoustic signal needs to be filtered and denoised, specifically using a Gaussian filtering method.

4. The intelligent action recognition method as described in claim 1, characterized in that, The specific steps for recognizing the action type are as follows: Based on the acoustic feature sequence, MFCC features are extracted, and based on the MFCC features and the historical MFCC features of different action types, the similarity between the acoustic feature sequence and different action types is obtained as the MFCC similarity. Based on the acoustic feature sequence, LPCC features are extracted, and based on the LPCC features and the historical LPCC features of different action types, the similarity between the acoustic feature sequence and different action types is obtained as the LPCC similarity. Determine whether the action type can be determined based on the MFCC similarity and the LPCC similarity. If yes, determine the action type based on the MFCC similarity and the LPCC similarity. If no, proceed to the next step. The overall similarity is determined based on the energy spectrum of the acoustic feature sequence, the MFCC similarity, and the LPCC similarity, and the action type is determined based on the overall similarity.

5. The intelligent action recognition method as described in claim 4, characterized in that, The MFCC similarity is determined based on the Mahalanobis distance between the MFCC features and the historical MFCC features of different action types.

6. The intelligent action recognition method as described in claim 1, characterized in that, When the action type is a hook-pull type, the specific steps for determining the fault type and fault probability of the initial action are as follows: Based on the diagnostic acoustic features, the short-time frame energy ratio and the truncated energy pulse width are obtained, and the fused acoustic temporal features are obtained based on the short-time frame energy ratio and the truncated energy pulse width. Based on the diagnostic acoustic features, the power spectral density and mean square frequency are obtained, and based on the power spectral density and mean square frequency, the fused acoustic frequency domain features are obtained. Based on the fused acoustic time-domain features and the fused acoustic frequency-domain features, a classification model based on the SVM algorithm is used to obtain the fault type of the initial action. Based on the fault type, the fused acoustic time-domain features, and the fused acoustic frequency-domain features, a prediction model is constructed to obtain the fault probability of the fault type.

7. The intelligent action recognition method as described in claim 1, characterized in that, The specific steps for determining the fault type and fault probability of the object to be detected are as follows: Determine whether the fault type of the confirmed action is consistent with the fault type of the initial action. If yes, proceed to the next step; otherwise, output that the fault type cannot be determined temporarily and further testing is required. The failure probability weight of the confirmation action is determined based on the number of operations for the recommended action type and the recommended action type; the failure probability weight of the initial action is determined based on the action type of the initial action. The failure probability of the object to be detected is determined based on the weight of the failure probability of the initial action and the failure probability of the confirmation action, as well as the failure probability of the initial action and the failure probability of the confirmation action. When the failure probability of the object to be detected is greater than a preset value, an alarm signal is output.

Images