A frog defect scanning and positioning method based on an acoustic sensing smart hammer

Abstract

The application provides a frog defect scanning and positioning method based on an acoustic sensing intelligent hammer, relates to the technical field of track structure detection, divides a to-be-detected frog into a plurality of grid units, and uses a hammer handle end to knock each grid unit in turn, and sends the collected knocking sound signals to a handheld mobile processing terminal; the handheld mobile processing terminal processes the knocking sound signals sent by the hammer handle end, and generates a defect probability distribution map of the to-be-detected frog; wherein, knocking response features are extracted from the knocking sound signals, the knocking response features are effectively determined, and finally the effective knocking response features corresponding to all the grid units are fused to generate the defect probability distribution map of the to-be-detected frog, and the probability and position of defects are determined based on the defect probability distribution map. Thus, quantitative analysis and accurate positioning of the frog are realized, and the objectivity and accuracy of detection are significantly improved compared with traditional detection methods.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of track structure detection, in particular to a frog defect scanning and positioning method based on an acoustic sensing intelligent hammer. BACKGROUND

[0002] Railway frogs are the core conversion devices of the track network, which realize the switching of trains between different tracks through dynamic guidance of the wheels, and maintain the geometric continuity of the tracks to ensure the efficiency of transportation. Although cast high manganese steel (CHMS) frogs have excellent work hardening performance, their structural discontinuity causes them to bear periodic impact loads of up to 200-350 kN when trains pass through. Under the long-term action of alternating stress, manufacturing defects (such as inclusions and pores) of the steel rail or micro-cracks generated during use are prone to expand in stress concentration areas such as the rail head and welded joints, ultimately leading to the risk of fracture and directly affecting the safety of railway operation.

[0003] Traditional hammering detection and positioning methods identify cracks by knocking on the surface of the track and analyzing the characteristics of the sound waves. Complete metal structures can produce clear harmonics due to differences in acoustic characteristics, while crack areas exhibit abnormal acoustic responses with low-frequency attenuation. Although this method has the advantages of simple operation and significant cost-effectiveness, its technical limitations are quite prominent: the detection results are highly dependent on the experience and subjective judgment of the operator, and false negatives can easily occur in complex sound field environments; the physical principle limits its ability to identify only surface cracks, with insufficient sensitivity to internal and deep defects in the steel rail; and due to the lack of a standardized acoustic evaluation system, it is impossible to establish a quantitative relationship between acoustic characteristics and crack parameters, resulting in long-term reliance on qualitative analysis for maintenance decisions.

[0004] Traditional penetration detection is a non-destructive testing method. It is mainly based on the principle of capillary action, which realizes defect visualization through the interaction of penetrant, developer, and cracks. This method uses a three-stage detection process: first, the penetrant is allowed to penetrate into the surface cracks, then the residual penetrant is removed, and the developer is applied to form a reverse capillary effect. Finally, the crack morphology is determined by visual or fluorescent observation, with the advantages of simple equipment, controllable cost, and high sensitivity in detecting open surface defects. However, its application has significant limitations: it can only identify cracks that penetrate the surface and is ineffective for closed and internal damage; strict pretreatment is required to ensure the cleanliness of the detection surface, otherwise interference signals may occur; there is a lack of quantitative correlation model between crack size and visualization characteristics; and extreme environmental conditions can change the physical parameters of the penetrant, affecting the reliability of the detection. These technical bottlenecks restrict its deep application in the field of precision detection.

[0005] It can be seen that the existing detection system relies on traditional methods such as hammering inspection and penetration detection. There are significant limitations: first, only surface or shallow defects can be identified, and there is a blind area for internal damage of special components; second, it relies on human experience and cannot quantitatively analyze defect size and expansion trend; third, the sensitivity is insufficient, and it is difficult to capture millimeter-level micro-cracks. Data shows that about 37% of the failure accidents of frog are caused by undetected internal defects, especially when the wheel-rail contact stress reaches 1500-2500 MPa, the crack propagation rate can reach 0.3 mm / 10,000 times of axle load, and more accurate detection means is urgently needed. SUMMARY

[0006] Therefore, the purpose of the present application is to provide a frog defect scanning and positioning method based on an acoustic sensing intelligent hammer, which can realize high-precision detection and quantitative evaluation of frog defects.

[0007] In a first aspect, the present application provides a frog defect scanning and positioning method based on an acoustic sensing intelligent hammer, the acoustic sensing intelligent hammer comprising a hammer handle end and a handheld mobile processing terminal; the method comprising the following steps:

[0008] Divide the frog to be inspected into a plurality of grid units, and use the hammer handle end to knock each of the grid units in turn, and send the collected knocking sound signals to the handheld mobile processing terminal;

[0009] Use the handheld mobile processing terminal to process the knocking sound signals sent by the hammer handle end to generate a defect probability distribution map of the frog to be inspected; wherein, knocking response features are first extracted from the knocking sound signals, then the knocking response features are effectively determined, and finally the effective knocking response features corresponding to all the grid units are fused to generate the defect probability distribution map of the frog to be inspected, and the probability and position of the defect are determined based on the defect probability distribution map.

[0010] In some embodiments, the hammer handle end comprises an acoustic sensor, an audio decoding chip and a communication unit; wherein, the acoustic sensor is used to collect the knocking sound signals when knocking each of the grid units in turn, the audio decoding chip is used to perform analog-to-digital conversion on the knocking sound signals, and finally the analog-to-digital converted knocking sound signals are sent to the handheld mobile processing terminal through the communication unit.

[0011] In some embodiments, the knocking response features are extracted from the knocking sound signals, comprising the following steps:

[0012] Performing short-time Fourier transform on the collected knocking sound signals;

[0013] Obtaining a mel spectrum through a mel filter bank for the knocking sound signals after short-time Fourier transform;

[0014] logarithmically transforming and discrete cosine transforming the mel spectrum to obtain mel frequency cepstral coefficients;

[0015] extracting the knocking response features from the mel frequency cepstral coefficients by using a convolutional neural network.

[0016] In some embodiments, the convolutional neural network for extracting the knocking response features from the mel frequency cepstral coefficients comprises four layers, and the number of channels and the convolution kernel are (1, 3x3), (64, 2x2), (64, 5x5), (64, 3x3) respectively; and the operation of the i-th layer of the convolutional neural network for extracting the knocking response features is as follows:

[0017]

[0018] wherein i = 1, 2, 3, 4, C in , are the input and output channel numbers respectively, * is a two-dimensional convolution operator, is a convolution kernel matrix, is the bias of the j-th output channel; x is the input mel frequency cepstral coefficient feature, and the final obtained knocking response feature vector is M = f conv,4 (f conv,3 (f conv,2 (f conv,1 (x))))).

[0019] In some embodiments, the effective determination of the knocking response features comprises the following steps:

[0020] If the knocking response features are determined to be effective, the knocking response features are stored;

[0021] If the knocking response features are determined to be ineffective, the corresponding grid unit is knocked again until all the knocking response features corresponding to the grid units are effective;

[0022] wherein the first multi-layer perception network is used to determine the effectiveness of the knocking response features, the first multi-layer perception network comprises one hidden layer, the activation function adopts ReLU, and the output layer is fitted by a Sigmoid function:

[0023] MPL valid (M) = Sigmoid(f linear,v2 (ReLU(f linear,v1 (M)))

[0024] wherein MPL valid (M) is a knocking effectiveness determination function, f linear , · (·) is a linear neural network; when MPL valid(M) is greater than a set threshold, it is determined that the corresponding knock response feature is valid.

[0025] In some embodiments, the valid determination of the knock response feature further comprises the following steps:

[0026] The obtained valid knock response feature is subjected to single-point defect probability prediction by using a second multi-layer perception network, the second multi-layer perception network comprises one hidden layer, the activation function adopts ReLU, and the output layer is fitted by a Sigmoid function:

[0027] MPL detection (M) = Sigmoid(f linear,d2 (ReLU(f linear,d1 (M)))

[0028] Wherein, MPL detection (M) is a single-point defect probability prediction function.

[0029] In some embodiments, the fusion of the valid knock response features corresponding to all the grid cells to generate the defect probability distribution map of the to-be-inspected frog joint comprises the following steps:

[0030] All the valid knock response features corresponding to all the grid cells are combined to form a feature matrix F;

[0031] The feature matrix is input into a third multi-layer perception network for calculation to obtain a probability distribution vector P joint ; wherein the third multi-layer perception network comprises multiple hidden layers, the activation functions of the intermediate layers all adopt ReLU, and the output layer is fitted by a Sigmoid function:

[0032] P joint = Simoid(f linear,j4 (ReLU(f linear,j3 (ReLU(f linear,j2 (ReLU(f linear,j1 (F))))))))

[0033] Based on the probability distribution vector P joint , a defect probability distribution map is generated, which is used to display the defect probability of each grid cell in the to-be-inspected frog joint through visual elements.

[0034] In a second aspect, the present application provides an acoustic sensing intelligent hammer, comprising a hammer handle end and a handheld mobile processing terminal, which is used to execute the steps of the frog joint defect scanning positioning method based on the acoustic sensing intelligent hammer according to any one of the first aspect.

[0035] In a third aspect, the present application provides an electronic device, comprising a processor, a memory and a bus, the memory storing machine readable instructions executable by the processor, when the electronic device is running, the processor and the memory communicate through the bus, the machine readable instructions are executed by the processor to perform the steps of the method for scanning and positioning defects of a frog based on an acoustic sensing smart hammer according to any one of the first aspect.

[0036] In a fourth aspect, the present application provides a computer readable storage medium, the computer readable storage medium storing a computer program, when the computer program is run by a processor, the steps of the method for scanning and positioning defects of a frog based on an acoustic sensing smart hammer according to any one of the first aspect are performed.

[0037] The method for scanning and positioning defects of a frog based on an acoustic sensing smart hammer according to the present application divides the frog to be detected into a plurality of grid units, and uses the hammer handle end to knock each of the grid units in turn, and sends the collected knocking sound signals to a handheld mobile processing terminal; the handheld mobile processing terminal processes the knocking sound signals sent by the hammer handle end to generate a defect probability distribution map of the frog to be detected; wherein the knocking response features are extracted from the knocking sound signals, the knocking response features are effectively determined, and finally the effective knocking response features corresponding to all the grid units are fused to generate the defect probability distribution map of the frog to be detected, and the probability and position of the defect are determined based on the defect probability distribution map. Thus, quantitative analysis and accurate positioning of the frog are realized, and the objectivity and accuracy of detection are significantly improved compared with the traditional detection method. BRIEF DESCRIPTION OF DRAWINGS

[0038] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed in the embodiments. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can also be obtained without creative labor.

[0039] Figure 1 The flow chart of the method for scanning and positioning defects of a frog based on an acoustic sensing smart hammer according to the embodiments of the present application is shown;

[0040] Figure 2 The structural schematic diagram of the hammer handle end according to the embodiments of the present application is shown;

[0041] Figure 3 The structural schematic diagram of the handheld mobile processing terminal according to the embodiments of the present application is shown;

[0042] Figure 4 The structural schematic diagram of the frog to be detected divided into a plurality of grid units according to the embodiments of the present application is shown;

[0043] Figure 5 A structural schematic diagram of a convolutional neural network for knock response feature extraction of mel-frequency cepstral coefficient features according to an embodiment of the present application is shown.

[0044] Figure 6 A structural schematic diagram of a third multi-layer perception network according to an embodiment of the present application is shown.

[0045] Figure 7 A structural schematic diagram of a defect probability distribution map according to an embodiment of the present application is shown.

[0046] Figure 8 A structural schematic diagram of an electronic device according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0047] To make the objectives, technical solutions, and superiorities of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described below in connection with the drawings in the embodiments of the present application. It should be understood that the drawings in the present application are only intended to illustrate and describe the present application, and are not intended to limit the scope of protection of the present application. In addition, it should be understood that the schematic drawings are not drawn according to the actual proportions. The flowcharts in the present application show the operations implemented according to some embodiments of the present application. It should be understood that the operations in the flowcharts can not be implemented in sequence, and the steps without logical context relationship can be reversed in sequence or implemented simultaneously. In addition, one or more other operations can be added to the flowcharts or one or more operations can be removed from the flowcharts under the guidance of the content of the present application.

[0048] In addition, the described embodiments are only some of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings herein can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed present application, but only represents selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0049] It should be noted that the term “comprise” will be used in the embodiments of the present application to indicate the presence of the features declared thereafter, but does not exclude the addition of other features.

[0050] In view of the technical problems proposed in the background art, the present application provides a method for scanning and positioning a frog defect based on an acoustic sensing intelligent hammer, which can realize high-precision detection and quantitative evaluation of frog defects.

[0051] In an embodiment, reference is made to the descriptionFigure 1 The application provides a frog defect scanning and positioning method based on an acoustic sensing intelligent hammer, the acoustic sensing intelligent hammer comprising a hammer handle end and a handheld mobile processing terminal; the method comprising the following steps:

[0052] S1, dividing the frog to be detected into a plurality of grid units, and using the hammer handle end to knock each of the grid units in turn, and sending the collected knocking sound signals to the handheld mobile processing terminal;

[0053] S2, using the handheld mobile processing terminal to process the knocking sound signals sent by the hammer handle end, to generate a defect probability distribution map of the frog to be detected; wherein, knocking response features are first extracted from the knocking sound signals, then the knocking response features are effectively determined, and finally the effective knocking response features corresponding to all the grid units are fused to generate the defect probability distribution map of the frog to be detected, and the probability and position of the defect are determined based on the defect probability distribution map.

[0054] In order to clearly understand the technical scheme of the embodiments of the application, the acoustic sensing intelligent hammer can be described first. Specifically, the acoustic sensing intelligent hammer comprises a hammer handle end and a handheld mobile processing terminal. Among them, the hammer handle end integrates a high-sensitivity acoustic sensor, a built-in audio decoding chip and a communication unit, realizes real-time collection and low-latency data transmission of knocking sound signals. The handheld mobile processing terminal systematically and intelligently analyzes the received data, and combines an optimized detection process to realize defect detection scanning and positioning. In an embodiment, as shown in the accompanying drawings, the hammer handle end integrates a high-precision shock-resistant MEMS acoustic sensor, and is built-in with a 24-bit high-precision audio decoding chip and a Bluetooth low-latency transmission unit, to ensure lossless transmission of knocking sound signals. In addition, it is also equipped with a lithium battery and a power management module to realize long-time endurance. As shown in the accompanying drawings, the handheld mobile processing terminal is equipped with an NPU computing architecture, receives the knocking sound signals sent by the hammer handle end through Bluetooth, specifically, the RK3588 processor NPU unit (computing power 6TOPS) can run a lightweight defect detection model in real time, and combines a mel filter bank to extract features; then the convolutional neural network is used to obtain the knocking response features, and the optimized work flow is used to obtain the grid knocking response features. In addition, the multi-node features are fused to obtain the defect probability distribution map. Finally, the defect type, probability distribution and spatial positioning are visually presented on the customized human-computer interface. And equipped with high energy density lithium polymer battery group, supporting continuous 10 hours operation, realizing high recognition accuracy and high positioning accuracy in complex working conditions, significantly better than traditional detection methods. Figure 2 Figure 3

[0055] ​​Specifically, the following will be described in detail how the step S2 utilizes the hand-held mobile processing terminal to process the knocking sound signal sent by the hammer handle end, and generates the defect probability distribution map of the to-be-inspected frog.

[0056] It should be noted that in the present application, the frog is divided into a plurality of grid units, and by knocking and subsequent processing of each grid unit, the defect can be located to a specific grid unit, achieving high-precision positioning; and through the division of the grid units, it is convenient to formulate a standardized knocking detection path (such as detecting row by row according to the grid order), avoiding missed detection or repeated detection, and improving the detection efficiency. In an embodiment, referring to the accompanying drawings Figure 4 According to the structural characteristics of the frog (such as the center rail, wing rail, frog heel, etc.), the specific area that needs to be knocked and detected can be determined, which usually covers the key positions prone to defects, and then a regular grid-shaped laser line is projected on the surface of the frog by a laser grid projection device, forming a visible grid reference line.

[0057] Firstly, the knocking response feature extraction is performed on the knocking sound signal sent by the hammer handle end. Specifically, the Mel-spectrum feature extraction is performed on the knocking sound signal sent by the hammer handle end; and then the convolutional neural network (CNN) is used to further extract and reduce the dimension of the Mel-spectrum feature, and learn more representative knocking response features.

[0058] When performing Mel-spectrum feature extraction, the Mel-frequency cepstrum coefficient (MFCC) method is adopted, which can effectively improve the resolution of low-frequency components and compress high-frequency components, so as to more accurately capture the frequency domain features of the hammering signal. The MFCC processing process is as follows: firstly, the knocking sound signal processed by the short-time Fourier transform (STFT) is obtained by a Mel filter bank to obtain a Mel spectrum, and then logarithmic transformation and discrete cosine transformation (DCT) are performed to remove the correlation in the feature vector, thereby extracting the main features. Finally, the feature normalization is performed to obtain more robust features. The MFCC feature can be expressed as:

[0059] X MFCC (τ,v),τ=1,2,…,N τ,MFCC ,v=1,2,…,N v,MFCC

[0060] wherein N τ,MFCC and N V,MFCC are the number of time and frequency subbands of MFCC, respectively.

[0061] Referring to the accompanying drawings Figure 5The convolutional neural network CNN for extracting the tap response feature from the mel-frequency cepstral coefficient feature includes four layers, and the mel-frequency cepstral coefficient feature is respectively subjected to a convolution network with a channel number and a convolution kernel of (1, 3x3), (64, 2x2), (64, 5x5), and (64, 3x3), and finally a K-dimensional tap response feature vector M is obtained. The convolutional neural network feature extraction operation of the i-th layer is as follows:

[0062]

[0063] wherein i = 1, 2, 3, 4, C in , are the input and output channel numbers respectively, * is a two-dimensional convolution operator, is a convolution kernel matrix, is the bias of the j-th output channel; x is the input mel-frequency cepstral coefficient feature, and thus we can obtain the tap response feature vector M = f coov,4 (f coov,3 (f conv,2 (f conv,1 (x))))).

[0064] After extracting the tap response feature by the convolutional neural network CNN, a multi-layer perceptron (MPL) is used to determine the validity of the tap. If the tap is determined to be valid, the extracted tap response feature M n is temporarily stored in the near-field memory of the device; if the tap is determined to be invalid, the corresponding grid cell is tapped again, and this step is repeated until the tap response features of all the grid cells are valid. This step aims to ensure that the tap signal meets the standard measurement specifications and effectively eliminates abnormal samples caused by improper tapping force, arm shaking, and other factors.

[0065] In an embodiment, a first multi-layer perceptron network is used to determine the validity of the tap response feature, the first multi-layer perceptron network including one hidden layer, the activation function using ReLU, and the output layer being fitted by a Sigmoid function:

[0066] MPL valid (M) = Sigmoid(f iinear,v2 (ReLU(f linear,v1 (M))))

[0067] wherein MPL valid (M) is a tap validity determination function, f linear,· (·) is a linear neural network, f linear,· (x) = W f x + b, W f is a linear weighting matrix, and b is a bias parameter. For example, the MPLvalid (M) is defined as the quality score of the tap, and a tap quality score threshold is set as 0.8. When the tap quality score output by the first multi-layer perception network is greater than or equal to 0.8, it is determined that the tap is valid, and the tap is stored. If the tap quality score is less than 0.8, a re-tap mechanism is triggered to remind the user to re-tap the corresponding grid unit until the tap is determined to be valid.

[0068] In another embodiment, while the first multi-layer perception network is used to effectively determine the tap response features, the second multi-layer perception network is further used to predict the single-point defect probability of the obtained valid tap response features. The second multi-layer perception network and the first multi-layer perception network have the same structure, contain one hidden layer, and use ReLU as the activation function. The output layer is fitted by a Sigmoid function:

[0069] MPL detection (M) = Sigmoid(f linear,d2 (ReLU(f linear,d1 (M))))

[0070] where MPL detection (M) is a single-point defect probability prediction function, which is used to represent the possibility of the presence of defects in the network unit corresponding to the current tap response feature. For example, a single-point defect probability threshold is set as 0.5. When the single-point defect probability output by the second multi-layer perception network is greater than or equal to 0.5, it is marked as a suspected defect area. In this way, the approximate situation of the defect probability in different areas of the frog can be initially understood.

[0071] After obtaining the valid tap response features corresponding to the grid units, the valid tap response features corresponding to all the grid units need to be fused, i.e., the neural network algorithm is used to fuse the valid tap response features M1, M2, …, M N of all the N grid units, and then the overall defect probability distribution map of the frog under test is jointly estimated.

[0072] In an embodiment, the valid tap response features extracted from each tap point are input into a third multi-layer perception network (MPL) to fuse the valid tap response features corresponding to all the grid units, and a deeper probability distribution map is obtained. Referring to the accompanying drawings, the third multi-layer perception network includes two hidden layers, and all the input tap response features can be written as an MxN-dimensional feature matrix F: Figure 6

[0073] F = [M1, …, M K ] T

[0074] An N-dimensional probability distribution vector P joinr ​In the middle layer of the neural network, the ReLU is selected as the activation function, and the Sigmoid function is selected as the activation function in the last layer to fit the multi-event probability distribution:

[0075] P joint = Simoid(f linear,j4 (ReLU(f linear,j3 (ReLU(f linear,j2 (ReLU(f iinear,j1 (F))))))))).

[0076] Further, according to the obtained probability distribution vector P joint , a defect probability distribution graph is generated to display the defect probability of each grid unit in the to-be-inspected frog through visual elements, that is, to convert the abstract defect probability distribution into intuitive graphical information. In an embodiment, the generated defect probability distribution graph is as shown in the accompanying drawings of the specification, so that the probability and position of defects can be quickly judged through the distribution of the defect probability distribution graph, and the efficiency and accuracy of the frog defect scanning and positioning based on the acoustic sensing intelligent hammer are significantly improved. Figure 7

[0077] In addition, it should be noted that, in the model quantization and deployment process of the present application, when the rknn-toolkit is used to quantize the model and deploy it to the NPU, the convolutional neural network CNN and the multi-layer perception network MPL will be fused into an end-to-end lightweight model, so that the calculation amount of the quantized model is reduced, the inference speed is improved, the real-time requirement of frog detection is met, and the power consumption of the NPU is reduced, which is suitable for the endurance requirement of handheld mobile processing terminals.

[0078] It can be seen that the frog defect scanning and positioning method based on the acoustic sensing intelligent hammer provided by the present application solves the problems of strong experience dependence, inability to quantify, and only surface defects can be identified in penetration detection by integrating a high-sensitivity acoustic sensor and an audio decoding chip at the end of the hammer handle, combining the Mel frequency cepstral coefficient (MFCC) feature extraction and the lightweight convolutional neural network model, and performing grid knocking detection on the frog to obtain a defect probability distribution graph through feature fusion, thereby realizing high-precision defect positioning and quantitative evaluation and significantly improving the detection accuracy and efficiency.

[0079] Based on the same inventive concept, the present application also provides an acoustic sensing intelligent hammer including a hammer handle end and a handheld mobile processing terminal, which is used to perform the steps of the above-mentioned frog defect scanning and positioning method based on the acoustic sensing intelligent hammer. Since the acoustic sensing intelligent hammer in the present application solves the problem by a similar principle to the above-mentioned frog defect scanning and positioning method based on the acoustic sensing intelligent hammer, the implementation of the acoustic sensing intelligent hammer can be referred to the implementation of the method, and the repeated parts will not be described again.​

[0080] Based on the same concept of the present application, the specification is attached Figure 8 As shown in the accompanying drawings, the electronic device 800 provided by the embodiments of the present application includes at least one processor 801, at least one network interface 804 or other user interface 803, a memory 805, and at least one communication bus 802. The communication bus 802 is used to realize the connection and communication between the components. The electronic device 800 can optionally include a user interface 803, including a display (for example, a touch screen, an LCD, a CRT, holographic imaging (Holographic), or a projector, etc.), a keyboard or a clicking device (for example, a mouse, a trackball, a touchpad, or a touch screen, etc.).

[0081] The memory 805 can include read-only memory and random access memory, and provide instructions and data for the processor 801. A part of the memory 805 can also include a non-volatile random access memory (NVRAM).

[0082] In some embodiments, the memory 805 stores the following elements, executable modules or data structures, or a subset of them, or an extended set of them:

[0083] The operating system 8051 includes various system programs, used to realize various basic services and process hardware-based tasks;

[0084] The application program module 8052 includes various application programs, such as a desktop (launcher), a media player (MediaPlayer), a browser (Browser), etc., used to realize various application services.

[0085] In the embodiments of the present application, by calling the programs or instructions stored in the memory 805, the processor 801 is used to execute the steps in the method for scanning and positioning the frog defect based on the acoustic sensing intelligent hammer, and high-precision detection and quantitative evaluation of the frog defect can be realized.

[0086] The present application also provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to perform the steps in the method for scanning and positioning the frog defect based on the acoustic sensing intelligent hammer.

[0087] Specifically, the storage medium can be a general storage medium, such as a mobile disk, a hard disk, etc., and the computer program stored on the storage medium can be executed to perform the above method for scanning and positioning the frog defect based on the acoustic sensing intelligent hammer.

[0088] In the embodiments of the present application, it should be understood that the disclosed apparatus and method can be implemented in other manners. The embodiments described above are merely exemplary, for example, the division of the units is only a logical function division, and there can be another division manner in actual implementation; for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections can be indirect couplings or communication connections through some interfaces, devices or units, and can be in electrical, mechanical or other forms.

[0089] The units described as separated components can or can not be physically separated, and the components displayed as units can or can not be physical units, i.e., can be located in one place, or can be distributed on a plurality of network units. In actual implementation, some or all of the units can be selected according to actual needs to achieve the purposes of the embodiments of the present application.

[0090] In addition, each functional unit in the embodiments of the present application can be integrated in one processing unit, or each unit can exist physically as a separate unit, or two or more units can be integrated in one unit.

[0091] If the functions are implemented in the form of software function units and sold or used as independent products, they can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the part that makes contributions to the prior art, or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods in the various embodiments of the present application. The foregoing storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disk, and various other media that can store program codes.

[0092] Finally, it should be noted that the above embodiments are merely specific implementations of this application, used to illustrate the technical solutions of this application, and not to limit them. The protection scope of this application is not limited thereto. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the scope of the technology disclosed in this application; and these modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application. All should be covered within the protection scope of this application. Therefore, the protection scope of this application should be determined by the protection scope of the claims.

Claims

1. A frog defect scanning and positioning method based on acoustic sensing smart hammer, characterized in that, The acoustic sensing smart hammer comprises a hammer handle end and a handheld mobile processing terminal; the method comprises the following steps: Divide the to-be-inspected frog into a plurality of grid units, and use the hammer handle end to knock each of the grid units in turn, and send the collected knocking sound signals to the handheld mobile processing terminal; the hammer handle end comprises an acoustic sensor, an audio decoding chip and a communication unit; wherein the acoustic sensor is used to collect the knocking sound signals when each of the grid units is knocked in turn, the audio decoding chip is used to perform analog-digital conversion on the knocking sound signals, and the knocking sound signals after analog-digital conversion are sent to the handheld mobile processing terminal through the communication unit; Use the handheld mobile processing terminal to process the knocking sound signals sent by the hammer handle end, and generate a defect probability distribution map of the to-be-inspected frog; wherein the knocking response features are extracted from the knocking sound signals, the knocking response features are effectively determined, and the effective knocking response features corresponding to all the grid units are fused to generate the defect probability distribution map of the to-be-inspected frog, and the probability and position of defects are determined based on the defect probability distribution map; The knocking response features are extracted from the knocking sound signals, comprising the following steps: performing short-time Fourier transform on the collected knocking sound signals; obtaining a mel spectrum through a mel filter bank for the knocking sound signals after short-time Fourier transform; performing logarithmic transformation and discrete cosine transformation on the mel spectrum to obtain mel frequency cepstral coefficients; and using a convolutional neural network to extract knocking response features from the mel frequency cepstral coefficient features; The effective determination of the knocking response features comprises the following steps: if the knocking response features are determined to be effective, the knocking response features are stored; if the knocking response features are determined to be ineffective, the corresponding grid unit is knocked again until the knocking response features corresponding to all the grid units are effective; wherein a first multi-layer perception network is used to determine the effectiveness of the knocking response features, the first multi-layer perception network contains one hidden layer, the activation function uses ReLU, and the output layer is fitted by a Sigmoid function: wherein, is a knocking effectiveness determination function, is a linear neural network; when the obtained is greater than a set threshold value, it is determined that the corresponding knocking response feature is effective.

2. The method according to claim 1, wherein, Wherein, The convolutional neural network for knock response feature extraction on the mel frequency cepstral coefficient features includes four layers, and the channel number and the convolution kernel are respectively , , , ; and the operation of the i-th layer convolutional neural network for extracting the knock response feature is as follows: wherein, = 1, 2, 3, 4, are the input and output channel numbers, respectively, and is a two-dimensional convolution operator, is a convolution kernel matrix, is the bias of the jth output channel; is the input mel-frequency cepstral coefficient feature, and the final knock response feature vector is .

3. The method according to claim 2, wherein, The effective determination of the knocking response features further comprises the following steps: A second multi-layer perception network is used to predict the single-point defect probability of the obtained effective knocking response features, the second multi-layer perception network contains one hidden layer, the activation function uses ReLU, and the output layer is fitted by a Sigmoid function: wherein, is the single point defect probability prediction function.

4. The method according to claim 3, wherein, The fusion of the effective knocking response features corresponding to all the grid units to generate the defect probability distribution map of the to-be-inspected frog comprises the following steps: combining the effective knock response features corresponding to all the grid cells into a feature matrix ; Input the feature matrix into a third multi-layer perception network for calculation to obtain a probability distribution vector ; wherein the third multi-layer perception network comprises a plurality of hidden layers, the intermediate layer activation functions all adopt ReLU, and the output layer is fitted through a Sigmoid function: based on the probability distribution vector generating a defect probability distribution map for presenting the defect probability of each grid cell in the inspected frog by a visual element.

5. An acoustic sensing smart hammer, characterized by, The acoustic sensing smart hammer comprises a hammer handle end and a handheld mobile processing terminal, which are used to perform the steps of the frog defect scanning and positioning method based on the acoustic sensing smart hammer in claim 1.

6. An electronic device, comprising: Comprise: A processor, a memory, and a bus, the memory storing machine readable instructions executable by the processor, when the electronic device is running, the processor and the memory communicating through the bus, the machine readable instructions being executed by the processor to perform the steps of the method for scanning and positioning the frog defect based on the acoustic sensing smart hammer according to any one of claims 1 to 4.

7. A computer readable storage medium characterized in that, The computer readable storage medium stores a computer program, the computer program being executed by the processor to perform the steps of the method for scanning and positioning the frog defect based on the acoustic sensing smart hammer according to any one of claims 1 to 4.

Images