An abnormal sound recognition method and device, electronic equipment and readable storage medium

Abstract

This invention discloses a method, device, electronic device, and computer-readable storage medium for identifying abnormal sounds. Applied to smart devices with external speaker functionality, it addresses the low efficiency and accuracy of existing abnormal sound identification methods. The invention proposes acquiring the sound signal played by the smart device to obtain a recording signal corresponding to the original audio signal; aligning the recording signal with the original audio signal to ensure their start times are the same; obtaining a recording spectrogram based on the aligned recording signal and a raw spectrogram based on the original audio signal; and comparing and analyzing the recording spectrogram and the raw spectrogram to identify the abnormal sound signal. This invention enables automatic identification of abnormal sound signals, improving identification efficiency and accuracy while reducing labor costs.

Description

Technical Field

[0001] This invention relates to the field of intelligent electronic device technology, and in particular to a method, apparatus, electronic device, and computer-readable storage medium for identifying unusual sounds. Background Technology

[0002] For voice interaction products with external speaker functionality, such as smart doorbells, the user version software of the external speaker function needs to be inspected before product delivery to determine if there are any abnormal sounds. This is usually done by listening manually. However, manual listening is limited by noise interference in the listening environment and the fatigue level of the listener, which has a significant impact on the accuracy of the listening results, affecting the efficiency and accuracy of abnormal sound recognition.

[0003] Therefore, how to improve the efficiency and accuracy of abnormal sound recognition and reduce labor costs has become a problem that needs to be solved by those skilled in the art. Summary of the Invention

[0004] The purpose of this invention is to provide a method, device, electronic device, and computer-readable storage medium for identifying abnormal sounds, which can automatically identify abnormal sound signals during use, thereby improving identification efficiency and accuracy and reducing labor costs.

[0005] To address the aforementioned technical problems, embodiments of the present invention provide a method for identifying unusual sounds, applied to smart devices with external speaker functionality. The method includes:

[0006] Acquire the sound signal played by the smart device to obtain a recording signal corresponding to the original audio signal;

[0007] Align the recording signal with the original audio signal so that the start time of the recording signal and the original audio signal are the same;

[0008] The recorded spectrogram is obtained from the signal-aligned recorded signal, and the original spectrogram is obtained from the original audio signal.

[0009] By comparing and analyzing the recorded spectrogram and the original spectrogram, abnormal sound signals are identified.

[0010] Optionally, aligning the recorded signal with the original audio signal so that their start times are the same includes:

[0011] The recording signal and the original audio signal are cross-correlated using a cross-correlation function to obtain the cross-correlation result.

[0012] The delay time of the recorded signal relative to the original audio signal is determined based on the cross-correlation results;

[0013] The recording signal is shifted backward along the time axis by the delay duration so that the start time of the recording signal and the original audio signal are the same.

[0014] Optionally, the cross-correlation function is:

[0015] Where c(iN) represents the i-th cross-correlation function value, N represents the data length of the original audio signal, M represents the data length of the recorded signal, i∈[1,M+N], x(i) represents the i-th signal in the original audio signal, and y(M-i+1) represents the M-i+1-th signal in the recorded signal;

[0016] The cross-correlation results include multiple cross-correlation function values.

[0017] Optionally, obtaining the recorded spectrogram from the signal-aligned recording signal and obtaining the original spectrogram from the original audio signal includes:

[0018] The order of the Fast Fourier Transform and the windowing function are preset, and the Fast Fourier Transform is performed on the aligned recording signal and the original audio signal respectively to obtain the frequency domain data corresponding to the aligned recording signal and the frequency domain data corresponding to the original audio signal.

[0019] A recording spectrogram is generated based on the frequency domain data corresponding to the subsequent recording signal;

[0020] A primitive spectrum is generated based on the frequency domain data corresponding to the original audio signal.

[0021] Optionally, the step of comparing and analyzing the recorded spectrogram and the original spectrogram to identify abnormal sound signals includes:

[0022] The recorded spectrogram is identified to determine the heterophonic audio segments;

[0023] The frequency points in the heterophonic audio bands in the spectrogram matrix corresponding to the recorded spectrogram and the original spectrogram are subjected to difference processing to obtain multiple frequency differences corresponding to each time.

[0024] The abnormal sound time is identified based on the multiple frequency differences corresponding to each time moment;

[0025] The signal corresponding to the time of the abnormal sound is identified as the abnormal sound signal.

[0026] Optionally, identifying the time of abnormal sound based on the multiple frequency differences corresponding to each time moment includes:

[0027] For each of the multiple frequency differences corresponding to each time point, it is determined whether the frequency difference is greater than a preset difference. If so, the target frequency point corresponding to the frequency difference is retained in the recording spectrogram.

[0028] For each time point, the frequency differences between each target frequency point corresponding to that time point are accumulated to obtain the accumulated frequency corresponding to that time point;

[0029] Each abnormal moment is determined based on the cumulative frequency corresponding to each of the aforementioned moments;

[0030] For each of the abnormal moments, the energy difference value of the abnormal moment is calculated based on the energy of each target frequency point corresponding to the abnormal moment;

[0031] Determine whether the energy difference value at the abnormal moment is less than a preset difference value. If so, determine that the abnormal moment is an abnormal sound moment.

[0032] Optionally, calculating the energy difference value at the abnormal time based on the energy of each target frequency point corresponding to the abnormal time includes:

[0033] Based on the energy at each target frequency point corresponding to the abnormal time, the energy difference value at the abnormal time is calculated using the energy difference relationship formula, wherein the energy difference relationship formula is:

[0034] Where S(t) represents the energy difference value at time t, L represents the total number of target frequency points at time t, X(j) represents the energy value of the j-th target frequency point at time t, and mean(X) represents the average energy value of the L target frequency points at time t.

[0035] This invention also provides an abnormal sound recognition device, applied to a smart device with external speaker functionality, the device comprising:

[0036] The acquisition module is used to acquire the sound signal played by the smart device and obtain a recording signal corresponding to the original audio signal;

[0037] An alignment module is used to align the recording signal with the original audio signal so that the start time of the recording signal and the original audio signal are the same.

[0038] The processing module is used to obtain the recorded spectrogram based on the signal-aligned recording signal, and to obtain the original spectrogram based on the original audio signal;

[0039] The identification module is used to compare and analyze the recorded spectrogram and the original spectrogram to identify abnormal sound signals.

[0040] This invention also provides an electronic device, comprising:

[0041] Memory, used to store computer programs;

[0042] A processor is used to implement the steps of the abnormal sound recognition method as described above when executing the computer program.

[0043] This invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the abnormal sound recognition method described above.

[0044] This invention provides a method, apparatus, electronic device, and computer-readable storage medium for identifying unusual sounds, applicable to smart devices with external speaker functionality. The method includes: acquiring a sound signal played by the smart device to obtain a recording signal corresponding to the original audio signal; aligning the recording signal with the original audio signal so that their start times are the same; obtaining a recording spectrogram based on the aligned recording signal and an original spectrogram based on the original audio signal; and comparing and analyzing the recording spectrogram and the original spectrogram to identify the time of unusual sound.

[0045] As can be seen, in this embodiment of the invention, the corresponding sound signal is obtained during the playback of the original audio signal by the smart device to obtain the corresponding recording signal. Then, the recording signal is aligned with the original audio signal so that the start time of the recording signal and the original audio signal are the same. The recording spectrogram is obtained based on the signal-aligned recording signal, and the original spectrogram is obtained based on the original audio signal. By comparing and analyzing the recording spectrogram and the original spectrogram, the time of abnormal sound can be identified, thereby identifying the abnormal sound signal corresponding to the time of abnormal sound. This invention can realize the automatic identification of abnormal sound signals, which is beneficial to improve the identification efficiency and accuracy and reduce labor costs. Attached Figure Description

[0046] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the prior art and embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0047] Figure 1 This is a flowchart illustrating a method for identifying unusual sounds provided in an embodiment of the present invention;

[0048] Figure 2 This is a schematic diagram illustrating the cross-correlation result between the original audio signal and the recorded signal, provided in an embodiment of the present invention.

[0049] Figure 3A waveform diagram of the original audio signal and the recorded signal after signal alignment is provided in an embodiment of the present invention;

[0050] Figure 4 This invention provides a primitive spectrogram corresponding to an original audio signal.

[0051] Figure 5 This invention provides a spectrogram of a recording signal corresponding to a recording signal.

[0052] Figure 6 A schematic diagram of the accumulation frequency provided in an embodiment of the present invention;

[0053] Figure 7 A schematic diagram of energy difference provided in an embodiment of the present invention;

[0054] Figure 8 This is a schematic diagram of the structure of an abnormal sound recognition device provided in an embodiment of the present invention;

[0055] Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention;

[0056] Figure 10 This is a schematic diagram of the structure of a computer-readable storage medium provided in an embodiment of the present invention. Detailed Implementation

[0057] This invention provides a method, device, electronic device, and computer-readable storage medium for identifying abnormal sounds. During use, these devices can automatically identify abnormal sound signals, thereby improving identification efficiency and accuracy and reducing labor costs.

[0058] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0059] Please refer to Figure 1 , Figure 1 This is a flowchart illustrating an abnormal sound recognition method provided in an embodiment of the present invention. The method is applied to a smart device with external speaker functionality and includes:

[0060] S110: Acquire the audio signal played by the smart device and obtain the recording signal corresponding to the original audio signal;

[0061] It should be noted that, in this embodiment of the invention, during the playback of a preset original audio signal on a smart device with an external speaker function, a microphone can be used to collect the sound signal played by the smart device to obtain a recording signal corresponding to the original audio signal.

[0062] S120: Align the recording signal with the original audio signal so that the start time of the recording signal and the original audio signal are the same;

[0063] Specifically, in this embodiment of the invention, after obtaining the recording signal, since there is a certain time delay between the recording signal and the original audio signal, it is necessary to align the recording signal with the original audio signal so that the start time of the recording signal is the same as the start time of the original audio signal, in order to ensure the accuracy of abnormal sound recognition.

[0064] S130: Obtain the recorded spectrogram based on the signal-aligned recording signal, and obtain the original spectrogram based on the original audio signal;

[0065] Specifically, after signal alignment, signal processing is performed on the aligned recording signal and the original audio signal to obtain the recording spectrogram corresponding to the aligned recording signal and the original spectrogram corresponding to the original audio signal.

[0066] S140: Compare and analyze the recorded spectrogram and the original spectrogram to identify abnormal sound signals.

[0067] It should be noted that after obtaining the spectrogram of the recording corresponding to the aligned recording signal and the original spectrogram of the original audio signal, the spectrogram of the recording corresponding to the aligned recording signal and the original spectrogram of the original audio signal are compared and analyzed to identify the abnormal sound signals that contain abnormal sounds.

[0068] Furthermore, the process of aligning the recorded signal with the original audio signal in S120 above, so that the start times of the recorded signal and the original audio signal are the same, may specifically include:

[0069] The cross-correlation function is used to perform a cross-correlation operation between the recorded signal and the original audio signal to obtain the cross-correlation result;

[0070] The delay time of the recorded signal relative to the original audio signal is determined based on the cross-correlation results;

[0071] The recording signal is shifted backward along the time axis by a delay time so that the start time of the recording signal and the original audio signal are the same.

[0072] It should be noted that in this embodiment of the invention, the recorded signal and the original audio signal can be cross-correlated using a cross-correlation function to obtain the cross-correlation result. Based on the cross-correlation result, the delay time of the recorded signal relative to the original audio signal can be determined. For example, if the original audio signal is x(n), where n = 1, 2…N, and the recorded signal for a single recording is y(m), where m = 1, 2…M, the cross-correlation function is:

[0073] Where c(iN) represents the i-th cross-correlation function value, N represents the data length of the original audio signal, M represents the data length of the recorded signal, i∈[1,M+N], x(i) represents the i-th signal in the original audio signal, and y(M-i+1) represents the M-i+1-th signal in the recorded signal;

[0074] Therefore, multiple cross-correlation function values ​​can be obtained based on the aforementioned cross-correlation function. These values ​​constitute the cross-correlation result, which determines the delay time of the recorded signal relative to the original audio signal. The recorded signal is then shifted backward along the time axis by this delay time to make the start time of the recorded signal and the original audio signal the same. The cross-correlation result is as follows: Figure 2 As shown, the alignment signal after aligning the recording signal and the original audio signal is as follows: Figure 3 As shown.

[0075] Furthermore, the process of obtaining the recorded spectrogram from the signal-aligned recording signal and the original spectrogram from the original audio signal in S130 above may specifically include:

[0076] The order of the Fast Fourier Transform and the windowing function are preset, and the Fast Fourier Transform is performed on the aligned recording signal and the original audio signal respectively to obtain the frequency domain data corresponding to the subsequent recording signal and the frequency domain data corresponding to the original audio signal.

[0077] A recording spectrogram is generated based on the frequency domain data corresponding to the subsequent recording signal;

[0078] Generate primitive spectrograms based on frequency domain data corresponding to the original audio signal.

[0079] It should be noted that in this embodiment of the invention, the FFT (Fast Fourier Transform) order and window function can be preset, the signal overlap rate can be set, and the aligned recording signal and the original audio signal can be sequentially subjected to FFT according to the overlap rate to obtain a set of frequency domain data corresponding to the subsequent recording signal and the original audio signal at different times (sampling time of the window function). A recording spectrogram is generated based on the frequency domain data corresponding to the subsequent recording signal, and a primitive spectrogram is generated based on the frequency domain data corresponding to the original audio signal. The primitive spectrogram is shown below. Figure 4 As shown, the spectrogram of the recording is as follows: Figure 5 As shown, rows in the spectrogram correspond to time, and columns correspond to frequency.

[0080] Furthermore, the process of comparing and analyzing the recorded spectrogram and the original spectrogram in S140 above to identify abnormal sound signals may specifically include:

[0081] Identify heterophonic audio segments from the recorded spectrograms;

[0082] The frequency points in the heterophonic bands in the spectrogram matrix corresponding to the recorded spectrogram and the original spectrogram are subjected to difference processing to obtain multiple frequency differences for each time.

[0083] The abnormal sound time is identified based on the multiple frequency differences corresponding to each time moment;

[0084] The signal corresponding to the time of the abnormal sound is identified as the abnormal sound signal.

[0085] It should be noted that, in the specific embodiments of the present invention, the recorded spectrogram can be identified. The spectrogram is a real number matrix, with multiple frequency points corresponding to each moment. For each frequency point at each moment, it is determined whether there are multiple consecutive frequency points whose frequencies are greater than a preset frequency value, thereby determining the heterophonic frequency segments F1-F2. The specific values ​​of F1 and F2 are determined according to the actual situation, and the embodiments of the present invention do not impose any special limitations on this.

[0086] Specifically, after identifying the different audio frequency bands, to further highlight the frequency bands with significant differences, the frequency points in the corresponding spectrogram matrices of the recorded and original spectrograms within the different audio frequency bands can be interpolated to obtain multiple frequency difference values ​​corresponding to each time point. That is, by subtracting the frequency values ​​of the corresponding frequency points in the recorded spectrogram at the same time point in F1-F2 from the corresponding frequency points in the original spectrogram, multiple frequency difference values ​​corresponding to that time point within the different audio frequency band F1-F2 can be obtained. Then, based on these multiple frequency difference values ​​for each time point, the time of the different audio frequency is identified, and the signal corresponding to that time point is determined as the different audio signal.

[0087] Furthermore, the process of identifying the time of abnormal sound based on the multiple frequency differences corresponding to each time moment can specifically include:

[0088] For each time point, determine whether the frequency band difference is greater than the preset difference. If so, retain the target frequency point corresponding to the frequency difference in the recording spectrogram.

[0089] For each time moment, the frequency differences of each target frequency point corresponding to that time moment are accumulated to obtain the accumulated frequency corresponding to that time moment;

[0090] Each abnormal moment is determined based on the cumulative frequency corresponding to each moment.

[0091] For each abnormal moment, the energy difference value at each target frequency point corresponding to the abnormal moment is calculated.

[0092] Determine whether the energy difference value at the abnormal moment is less than the preset difference value. If so, determine that the abnormal moment is the moment of abnormal sound.

[0093] It should be noted that, for each time point, the frequency differences in the different frequency bands F1 to F2 at that time are compared with a preset difference. If the frequency difference is greater than the preset difference, the frequency point corresponding to that frequency difference is retained in the recording spectrogram, and the retained frequency point is used as the target frequency point. If the frequency difference is not greater than the preset difference, the corresponding frequency point is not retained in the recording spectrogram. Thus, for each time point in F1-F2, the frequency differences corresponding to each target frequency point retained at that time can be accumulated to obtain the accumulated frequency corresponding to that time, thereby obtaining the accumulated frequency corresponding to each time point in the different frequency bands (e.g., ...). Figure 6 (As shown).

[0094] Specifically, after obtaining the accumulated frequency corresponding to each moment, each accumulated frequency can be further compared with a preset frequency range. The moment corresponding to the accumulated frequency value within the preset frequency range is determined as the abnormal sound moment, thus identifying each abnormal moment. After identifying each abnormal sound moment, for each abnormal moment, the energy difference value of the abnormal moment can be calculated based on the energy of each target frequency point corresponding to that abnormal moment (e.g., ...). Figure 7 As shown in the figure, it is further determined whether the energy difference value at the abnormal moment is less than the preset difference value. If it is less than the preset difference value, the abnormal moment is determined to be the abnormal sound moment, and the signal corresponding to the abnormal sound moment is the abnormal sound signal.

[0095] Furthermore, the process of calculating the energy difference value at the anomalous moment based on the energy of each target frequency point corresponding to the anomalous moment can specifically include:

[0096] Based on the energy at each target frequency point corresponding to the anomaly, the energy difference value at the anomaly is calculated using the energy difference relationship formula, where the energy difference relationship formula is:

[0097] Where S(t) represents the energy difference value at time t, L represents the total number of target frequency points at time t, X(j) represents the energy value of the j-th target frequency point at time t, and mean(X) represents the average energy value of the L target frequency points at time t.

[0098] It should be noted that the difference between the original audio signal and the recorded signal may be due to ambient background noise or interference signals collected during the testing process due to the elimination or simplification of the test shielding box to reduce costs. Therefore, in order to prevent these signals that are not abnormal sounds from being judged as abnormal sounds by the machine, the present invention can perform statistical processing on the signals at the target frequency points in the F1-F2 frequency band. Specifically, this can be achieved through the energy difference relationship formula. Calculate the energy difference value at each moment. If the energy difference value is less than the preset difference value (e.g., 2), it can be determined that the difference is stable in the frequency band F1 to F2 at that moment, that is, it can be determined that the difference is caused by the vertical bright line of the abnormal sound, that is, the abnormal sound signal is identified; otherwise, even if the energy difference is large, it is not judged as an abnormal sound, thereby improving the accuracy of abnormal sound identification.

[0099] As can be seen, in this embodiment of the invention, the corresponding sound signal is obtained during the playback of the original audio signal by the smart device to obtain the corresponding recording signal. Then, the recording signal is aligned with the original audio signal so that the start time of the recording signal and the original audio signal are the same. The recording spectrogram is obtained based on the signal-aligned recording signal, and the original spectrogram is obtained based on the original audio signal. By comparing and analyzing the recording spectrogram and the original spectrogram, the time of abnormal sound can be identified, thereby identifying the abnormal sound signal corresponding to the time of abnormal sound. This invention can realize the automatic identification of abnormal sound signals, which is beneficial to improve the identification efficiency and accuracy and reduce labor costs.

[0100] Based on the above embodiments, this invention also provides an abnormal sound recognition device, applied to smart devices with external speaker functionality. Please refer to [link / reference needed] for details. Figure 8 The device includes:

[0101] The acquisition module 11 is used to acquire the sound signal played by the smart device and obtain the recording signal corresponding to the original audio signal;

[0102] Alignment module 12 is used to align the recording signal with the original audio signal so that the start time of the recording signal and the original audio signal are the same;

[0103] Processing module 13 is used to obtain the recorded spectrogram based on the signal-aligned recording signal and to obtain the original spectrogram based on the original audio signal.

[0104] The recognition module 14 is used to compare and analyze the recorded spectrogram and the original spectrogram to identify the time of abnormal sound.

[0105] It should be noted that the abnormal sound recognition device provided in the embodiments of the present invention has the same beneficial effects as the abnormal sound recognition method provided in the above embodiments. For a detailed description of the abnormal sound recognition method involved in the embodiments of the present invention, please refer to the above embodiments, and this application will not repeat it here.

[0106] Figure 9 A structural diagram of an electronic device provided in an embodiment of this application, such as... Figure 9 As shown, the electronic device includes: a memory 20 for storing computer programs;

[0107] The processor 21 is used to implement the steps of the abnormal sound recognition method as described in the above embodiments when executing a computer program.

[0108] The electronic devices provided in this embodiment may include, but are not limited to, smartphones, tablets, laptops, or desktop computers.

[0109] The processor 21 may include one or more processing cores, such as a quad-core processor or an octa-core processor. The processor 21 may be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 21 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, the processor 21 may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, the processor 21 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.

[0110] The memory 20 may include one or more computer-readable storage media, which may be non-transitory. The memory 20 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In this embodiment, the memory 20 is used to store at least the following computer program 201, which, after being loaded and executed by the processor 21, is capable of implementing the relevant steps of the abnormal sound recognition method disclosed in any of the foregoing embodiments. In addition, the resources stored in the memory 20 may also include an operating system 202 and data 203, and the storage method may be temporary or permanent storage. The operating system 202 may include Windows, Unix, Linux, etc. The data 203 may include, but is not limited to, set offsets.

[0111] In some embodiments, the electronic device may further include a display screen 22, an input / output interface 23, a communication interface 24, a power supply 25, and a communication bus 26.

[0112] Those skilled in the art will understand that Figure 9 The structures shown do not constitute a limitation on electronic devices and may include more or fewer components than those shown.

[0113] It is understood that if the abnormal sound recognition method in the above embodiments is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and executes all or part of the steps of the methods in the various embodiments of this application. The aforementioned storage medium includes: USB flash drive, mobile hard drive, read-only memory (ROM), random access memory (RAM), electrically erasable programmable ROM, register, hard disk, removable disk, CD-ROM, magnetic disk, or optical disk, and other media capable of storing program code.

[0114] Based on this, such as Figure 10 As shown, this embodiment of the invention also provides a computer-readable storage medium 30, on which a computer program 31 is stored. When the computer program 31 is executed by a processor, it implements the steps of the above-described abnormal sound recognition method.

[0115] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0116] It should also be noted that, in this specification, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0117] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0118] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0119] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for identifying unusual sounds, characterized in that, The method, applied to smart devices with external speaker functionality, includes: Acquire the sound signal played by the smart device to obtain a recording signal corresponding to the original audio signal; Align the recording signal with the original audio signal so that the start time of the recording signal and the original audio signal are the same; The recorded spectrogram is obtained from the signal-aligned recorded signal, and the original spectrogram is obtained from the original audio signal. By comparing and analyzing the recorded spectrogram and the original spectrogram, abnormal sound signals are identified; wherein: The step of comparing and analyzing the recorded spectrogram and the original spectrogram to identify abnormal sound signals includes: identifying the recorded spectrogram to determine the abnormal sound segment; performing difference processing on each frequency point in the spectrogram matrix corresponding to the recorded spectrogram and the original spectrogram to obtain multiple frequency difference values ​​corresponding to each time moment; identifying the abnormal sound moment based on the multiple frequency difference values ​​corresponding to each time moment; and determining the signal corresponding to the abnormal sound moment as the abnormal sound signal. The step of identifying the abnormal audio segment by recognizing the audio spectrogram includes: for each frequency point at each moment in the audio spectrogram, determining whether there are multiple consecutive frequency points with frequencies greater than a preset frequency value, so as to identify the abnormal audio segment. The step of identifying abnormal sound moments based on multiple frequency differences corresponding to each moment includes: determining whether the frequency difference is greater than a preset difference for each of the multiple frequency differences corresponding to each moment; if so, retaining the target frequency point corresponding to the frequency difference in the recording spectrogram; for each moment, accumulating the frequency differences of each target frequency point corresponding to the moment to obtain the accumulated frequency corresponding to the moment; determining each abnormal moment based on the accumulated frequency corresponding to each moment; for each abnormal moment, calculating the energy difference value of the abnormal moment based on the energy of each target frequency point corresponding to the abnormal moment; determining whether the energy difference value of the abnormal moment is less than a preset difference value; if so, determining the abnormal moment as an abnormal sound moment. The step of determining each abnormal moment based on the accumulated frequency corresponding to each moment includes: after obtaining the accumulated frequency corresponding to each moment, comparing each accumulated frequency with a preset frequency range, and determining the moment corresponding to the accumulated frequency value within the preset frequency range as the abnormal sound moment.

2. The method for identifying unusual sounds according to claim 1, characterized in that, The step of aligning the recorded signal with the original audio signal so that the start times of the recorded signal and the original audio signal are the same includes: The recording signal and the original audio signal are cross-correlated using a cross-correlation function to obtain the cross-correlation result. The delay time of the recorded signal relative to the original audio signal is determined based on the cross-correlation results; The recording signal is shifted backward along the time axis by the delay duration so that the start time of the recording signal and the original audio signal are the same.

3. The method for identifying unusual sounds according to claim 1, characterized in that, The process of obtaining the recorded spectrogram based on the signal-aligned recorded signal and obtaining the original spectrogram based on the original audio signal includes: The order of the Fast Fourier Transform and the windowing function are preset, and the Fast Fourier Transform is performed on the aligned recording signal and the original audio signal respectively to obtain the frequency domain data corresponding to the aligned recording signal and the frequency domain data corresponding to the original audio signal. A recording spectrogram is generated based on the frequency domain data corresponding to the aligned recording signal. A primitive spectrum is generated based on the frequency domain data corresponding to the original audio signal.

4. The method for identifying unusual sounds according to claim 1, characterized in that, The calculation of the energy difference value at the abnormal time based on the energy of each target frequency point corresponding to the abnormal time includes: Based on the energy at each target frequency point corresponding to the abnormal time, the energy difference value at the abnormal time is calculated using the energy difference relationship formula, wherein the energy difference relationship formula is: ,in, L represents the energy difference at time t, and L represents the total number of target frequency points at time t. This represents the energy value of the j-th target frequency point at time t. This represents the average energy value of L target frequency points at time t.

5. A device for identifying unusual sounds, characterized in that, The device is applied to smart devices with external speaker functionality, and includes: The acquisition module is used to acquire the sound signal played by the smart device and obtain a recording signal corresponding to the original audio signal; An alignment module is used to align the recording signal with the original audio signal so that the start time of the recording signal and the original audio signal are the same. The processing module is used to obtain the recorded spectrogram based on the signal-aligned recording signal, and to obtain the original spectrogram based on the original audio signal; The identification module is used to compare and analyze the recorded spectrogram and the original spectrogram to identify abnormal sound signals; wherein: The step of comparing and analyzing the recorded spectrogram and the original spectrogram to identify abnormal sound signals includes: identifying the recorded spectrogram to determine the abnormal sound segment; performing difference processing on each frequency point in the spectrogram matrix corresponding to the recorded spectrogram and the original spectrogram to obtain multiple frequency difference values ​​corresponding to each time moment; identifying the abnormal sound moment based on the multiple frequency difference values ​​corresponding to each time moment; and determining the signal corresponding to the abnormal sound moment as the abnormal sound signal. The step of identifying the abnormal audio segment by recognizing the audio spectrogram includes: for each frequency point at each moment in the audio spectrogram, determining whether there are multiple consecutive frequency points with frequencies greater than a preset frequency value, so as to identify the abnormal audio segment. The step of identifying abnormal sound moments based on multiple frequency differences corresponding to each moment includes: determining whether the frequency difference is greater than a preset difference for each of the multiple frequency differences corresponding to each moment; if so, retaining the target frequency point corresponding to the frequency difference in the recording spectrogram; for each moment, accumulating the frequency differences of each target frequency point corresponding to the moment to obtain the accumulated frequency corresponding to the moment; determining each abnormal moment based on the accumulated frequency corresponding to each moment; for each abnormal moment, calculating the energy difference value of the abnormal moment based on the energy of each target frequency point corresponding to the abnormal moment; determining whether the energy difference value of the abnormal moment is less than a preset difference value; if so, determining the abnormal moment as an abnormal sound moment. The step of determining each abnormal moment based on the accumulated frequency corresponding to each moment includes: after obtaining the accumulated frequency corresponding to each moment, comparing each accumulated frequency with a preset frequency range, and determining the moment corresponding to the accumulated frequency value within the preset frequency range as the abnormal sound moment.

6. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the steps of the abnormal sound recognition method as described in any one of claims 1 to 4 when executing the computer program.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the abnormal sound recognition method as described in any one of claims 1 to 4.

Images