Abnormal audio detection method and device, electronic equipment, computer readable storage medium and computer program product

Abstract

The invention provides an abnormal audio detection method and device, electronic equipment, a computer readable storage medium and a computer program product. The method comprises the steps of determining target area audio data in an audio stream by detecting an audio amplitude change trend of the audio stream; performing at least one data statistics according to the target area audio data, and determining a target area audio feature corresponding to the target area audio data; and carrying out at least one detection of abnormal audio feature matching in a preset feature library and feature comparison of adjacent audio region data on the target region audio features, and determining an abnormal detection result of the target region audio features. According to the invention, the real-time performance and accuracy of audio detection can be improved.

Description

Technical Field

[0001] The present application relates to artificial intelligence technology, and in particular to an abnormal audio detection method, device, electronic device, computer-readable storage medium and computer program product. Background Art

[0002] At present, some smart devices, such as smart devices that support voice interaction, usually use network models based on machine learning to detect anomalies in the collected audio data. For example, audio data is converted into images, such as spectrograms for image recognition detection; or audio sampling data is converted into feature tensor data for detection, etc. This detection method requires the collection of a large number of labeled anomaly samples in advance to train the network model. However, it is difficult to collect enough samples for some audio anomalies with low probability of occurrence, which will lead to insufficient training of the network model and missed detection during actual detection, reducing the accuracy of the anomaly detection results. In addition, the network model needs to occupy a large amount of computing resources during operation, and the detection speed is slow. Therefore, it is difficult to perform continuous anomaly detection on the real-time audio stream, and may miss the anomaly detection of some audio data in the real-time audio stream, thereby reducing the real-time and accuracy of audio anomaly detection. Summary of the invention

[0003] The present application provides an abnormal audio detection method, device, electronic device, computer-readable storage medium and computer program product, which can improve the real-time and accuracy of audio detection.

[0004] The technical solution of this application is implemented as follows:

[0005] The present application provides a method for detecting abnormal audio, the method comprising:

[0006] Determine the target area audio data in the audio stream by detecting the audio amplitude change trend of the audio stream;

[0007] Perform at least one data statistic based on the target area audio data to determine the target area audio features corresponding to the target area audio data;

[0008] The target area audio feature is subjected to at least one of abnormal audio feature matching in a preset feature library and feature comparison of adjacent audio area data to determine an abnormality detection result of the target area audio feature.

[0009] The embodiment of the present application provides an abnormal audio detection device, comprising:

[0010] A detection module, configured to determine target area audio data in the audio stream by detecting a change trend of audio amplitude of the audio stream;

[0011] A statistical module, configured to perform at least one data statistic based on the target area audio data to determine a target area audio feature corresponding to the target area audio data;

[0012] The determination module is used to perform at least one of abnormal audio feature matching of a preset feature library and feature comparison of adjacent audio area data on the target area audio feature to determine the abnormality detection result of the target area audio feature.

[0013] Optionally, the determination module is also used to match the target area audio features with abnormal audio features in a preset feature library to determine the abnormality detection results corresponding to the target area audio data; the abnormal audio features are obtained by performing at least one data statistic on at least one abnormal sample audio data corresponding to at least one abnormal type.

[0014] Optionally, the determination module is further used to perform data feature comparison based on the target area audio features and adjacent audio area data corresponding to the target area audio data, to determine an abnormality detection result corresponding to the target area audio data.

[0015] Optionally, the determination module is also used to match the target area audio features with abnormal audio features in a preset feature library to determine a first abnormality detection result corresponding to the target area audio data; when the first abnormality detection result indicates that the target area audio features are not abnormal audio data, data features are compared based on the target area audio features and adjacent audio area data corresponding to the target audio area data to determine the abnormality detection result corresponding to the target area audio data.

[0016] Optionally, the determination module is also used to perform data feature comparison based on the target area audio feature and the adjacent audio area data corresponding to the target audio area data, to determine a second abnormality detection result corresponding to the target area audio data; when the second abnormality detection result indicates that the target area audio feature is not abnormal audio data, the target area audio feature is matched with the abnormal audio feature in a preset feature library to determine the abnormality detection result corresponding to the target area audio data.

[0017] Optionally, the determination module is also used to match the target area audio features with abnormal audio features in a preset feature library to determine a first abnormality detection result corresponding to the target area audio data, and to perform data feature comparison based on the target area audio features and adjacent audio area data corresponding to the target audio area data to determine a second abnormality detection result corresponding to the target area audio data; determine the abnormality detection result of the target audio features based on the first abnormality detection result and / or the second abnormality detection result.

[0018] Optionally, the determination module is also used to determine that the target area audio data is not abnormal audio data when both the first abnormality detection result and the second abnormality detection result indicate that the target area audio data is not abnormal audio data; and to determine that the target area audio data is abnormal audio data when at least one of the first abnormality detection result and the second abnormality detection result indicates that the target area audio data is abnormal audio data.

[0019] Optionally, the abnormal audio detection device also includes a construction module, which is used to obtain at least one abnormal sample audio data corresponding to each abnormal type in at least one abnormal type; for each abnormal sample audio data in the at least one abnormal sample audio data, at least one data statistics is performed according to the audio amplitude data in each abnormal sample audio data, and at least one feature statistic value corresponding to each abnormal sample audio data is determined; according to the at least one feature statistic value and at least one of the audio amplitude data, the abnormal audio feature corresponding to each abnormal sample audio data is determined; according to the abnormal audio feature corresponding to each abnormal sample audio data in the at least one abnormal sample audio data, at least one abnormal audio feature corresponding to each abnormal type is determined, thereby determining the preset feature library.

[0020] Optionally, the construction module is further used to determine at least one characteristic statistical value range based on the at least one characteristic statistical value and the preset fluctuation amount; and determine the abnormal audio feature corresponding to each abnormal sample audio data based on the at least one characteristic statistical value range and at least one of the audio amplitude data.

[0021] Optionally, the at least one data statistic includes at least one of the following:

[0022] Mean, standard deviation, and first-order difference calculations.

[0023] Optionally, the detection module is further used to slide a window of a preset length in the audio stream with a preset step size to obtain regional audio data corresponding to each sliding of the window; calculate the audio amplitude change trend of the regional audio data to determine the change trend characteristics corresponding to the regional audio data; and determine the regional audio data as the target regional audio data when the change trend characteristics corresponding to the regional audio data meet preset conditions.

[0024] Optionally, the regional audio data includes: audio amplitude data corresponding to the regional sampling points; the detection module is further used to determine the change trend characteristics corresponding to the regional audio data by performing first-order difference or second-order difference calculation on the audio amplitude data corresponding to the regional sampling points.

[0025] Optionally, the preset conditions include:

[0026] Among a preset number of change trend features obtained by sliding window detection, the change trend features of the regional audio data are the smallest or largest n change trend features, where n is greater than or equal to 1.

[0027] Optionally, the target area audio feature includes: an average value corresponding to audio amplitude data in the target area audio data; the adjacent audio area data corresponding to the target audio area data includes: first area audio data corresponding to a previous window of the target area audio data, and second area audio data corresponding to a subsequent window of the target area audio data; the data feature comparison based on the target area audio feature and the adjacent audio area data corresponding to the target audio area data includes: acquiring the first area audio data and the second area audio data; calculating a first average value corresponding to the audio amplitude data in the first area audio data and a second average value corresponding to the audio amplitude data in the second area audio data; determining an abnormality detection result corresponding to the target area audio data based on the first average value, the second average value and the average value corresponding to the audio amplitude data in the target area audio data.

[0028] Optionally, the determination module is further used to determine that the target area audio data is abnormal audio data when the ratio of the first average value to the average value corresponding to the audio amplitude data in the target area audio data is greater than a preset ratio threshold, and the ratio of the second average value to the average value corresponding to the audio amplitude data in the target area audio data is greater than the preset ratio threshold.

[0029] Optionally, the detection module is further used to obtain a mono audio stream in the original audio stream; and downsample the mono audio stream to determine the audio stream.

[0030] Optionally, the abnormal audio detection device also includes an update module, which is used to obtain at least one abnormal detection result obtained by performing abnormal detection on at least one real-time audio stream, and at least one annotation information corresponding to the at least one abnormal detection result; and update the abnormal audio features in the preset feature library according to the at least one annotation information corresponding to the at least one abnormal detection result.

[0031] The present application provides an electronic device, the electronic device comprising:

[0032] A memory for storing computer executable instructions;

[0033] The processor is used to implement the abnormal audio detection method provided in the embodiment of the present application when executing the computer executable instructions stored in the memory.

[0034] The present application provides a computer-readable storage medium storing a computer program or a computer-executable instruction for implementing the abnormal audio detection method provided by the present application when executed by a processor.

[0035] The present application provides a computer program product, including a computer program or a computer executable instruction. When the computer program or the computer executable instruction is executed by a processor, the abnormal audio detection method provided by the present application is implemented.

[0036] This application has the following beneficial effects:

[0037] By detecting the audio amplitude change trend of the audio stream, the target area audio data that may be abnormal in the audio stream is determined; at least one data statistic is performed based on the target area audio data to determine the target area audio features corresponding to the target area audio data; the target area audio features are matched with the abnormal audio features in the preset feature library, and / or, based on the target area audio features and the adjacent audio area data corresponding to the target audio area data, the abnormal detection result of the target area audio data is determined. In this way, the data features of the audio data are calculated in real time by means of data statistics and matched with the preset feature library, and / or, audio anomaly detection is achieved by comparing the features of the target area audio data with the features of the adjacent audio area data. The algorithm has low complexity and small amount of calculation, so that the audio stream can be continuously and automatically detected in real time without missing any audio data in the real-time audio stream, thereby improving the real-time and accuracy of audio detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 This is an optional flowchart of the abnormal audio detection method provided in the embodiment of the present application;

[0039] Figure 2This is an optional flowchart of the abnormal audio detection method provided in the embodiment of the present application;

[0040] Figure 3 This is an optional flowchart of the abnormal audio detection method provided in the embodiment of the present application;

[0041] Figure 4 This is an optional flowchart of the abnormal audio detection method provided in the embodiment of the present application;

[0042] Figure 5 This is a schematic diagram of an audio stream containing abnormal audio data provided by an embodiment of the present application Figure 1 ;

[0043] Figure 6 This is a schematic diagram of an audio stream containing abnormal audio data provided by an embodiment of the present application Figure 2 ;

[0044] Figure 7 This is an optional flowchart of the abnormal audio detection method provided in the embodiment of the present application;

[0045] Figure 8 This is an optional flowchart of the abnormal audio detection method provided in the embodiment of the present application;

[0046] Fig. 9 This is an optional flowchart of the abnormal audio detection method provided in the embodiment of the present application;

[0047] Fig.10 This is an optional flowchart of the abnormal audio detection method provided in the embodiment of the present application;

[0048] Fig.11 This is a schematic diagram of an audio stream containing abnormal audio data provided by an embodiment of the present application Figure 3 ;

[0049] Fig.12 It is a schematic diagram of the module processing process of the abnormal audio detection system provided in an embodiment of the present application;

[0050] Fig.13 It is a process diagram of an abnormal audio detection method applied to an actual scenario provided by an embodiment of the present application;

[0051] Fig.14 This is an optional structural diagram of an abnormal audio detection device provided in an embodiment of the present application;

[0052] Fig.15 It is a schematic diagram of an optional structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0053] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of this application.

[0054] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it can be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0055] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program with a predetermined function, and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories) or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.

[0056] Unless otherwise defined, all technical and scientific terms used in the embodiments of the present application have the same meanings as those commonly understood by those skilled in the art. The terms used in the embodiments of the present application are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0057] The relevant data collection and processing in the embodiments of this application should be strictly in accordance with the requirements of relevant national laws and regulations when applied in examples, obtain the informed consent or separate consent of the personal information subject, and carry out subsequent data use and processing within the scope of authorization of laws and regulations and the personal information subject.

[0058] Before further describing the embodiments of the present application in detail, the nouns and terms involved in the embodiments of the present application are explained. The nouns and terms involved in the embodiments of the present application are subject to the following interpretations.

[0059] At present, some smart devices, such as smart devices that support voice interaction, usually use network models based on machine learning to detect anomalies in the collected audio data. For example, audio data is converted into images, such as spectrograms for image recognition detection; or audio sampling data is converted into feature tensor data for detection, etc. This detection method requires the collection of a large number of labeled anomaly samples in advance to train the network model. However, it is difficult to collect enough samples for some audio anomalies with low probability of occurrence, which will lead to insufficient training of the network model and missed detection during actual detection, reducing the accuracy of the anomaly detection results. In addition, the network model needs to occupy a large amount of computing resources during operation, and the detection speed is slow. Therefore, it is difficult to perform continuous anomaly detection on the real-time audio stream, and may miss the anomaly detection of some audio data in the real-time audio stream, thereby reducing the real-time and accuracy of audio anomaly detection.

[0060] Alternatively, some audio anomaly detection methods have relatively complex judgment logic and require comprehensive judgment based on information from reference devices, resulting in poor usability.

[0061] The embodiments of the present application provide an abnormal audio detection method, device, electronic device, computer-readable storage medium and computer program product, which can continuously perform real-time automatic detection of audio streams, and improve the real-time and accuracy of audio detection. The abnormal audio detection method of the embodiments of the present application is applied to electronic devices. In some embodiments, the electronic device may include various types of user terminals such as laptops, tablet computers, desktop computers, smart home appliances, mobile smart devices (such as mobile phones, vehicle-mounted terminals, intelligent voice interaction devices), and may also include servers. Exemplarily, the server may include an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud services, network services, and basic cloud computing services such as big data and artificial intelligence platforms.

[0062] See also Figure 1 , Figure 1 The flowchart of the abnormal audio detection method provided in the embodiment of the present application, including S101 to S103, is as follows:

[0063] S101. Determine target area audio data in an audio stream by detecting a change trend of audio amplitude of the audio stream.

[0064] In S101, the audio stream may include a real-time collected audio stream or a historically recorded audio stream. The audio data of the audio stream includes the audio amplitude at each sampling point corresponding to the sampling frequency. By performing data statistics on the change trend of the audio amplitude at each sampling point of the audio stream, a sampling area where anomalies may occur can be determined from the audio stream, so as to further detect the audio data of the target area in the sampling area.

[0065] The target area audio data may include area audio data corresponding to one sampling area in the audio stream, or may include area audio data corresponding to multiple sampling areas.

[0066] In some embodiments, a mono audio stream in the original audio stream is obtained; the mono audio stream is downsampled to determine the real-time audio stream. In other words, the audio stream can be obtained by downsampling the mono data in the original audio stream. In this way, the amount of calculation and the amount of data transmission can be reduced, the calculation speed can be greatly improved, and the real-time performance of anomaly detection can be further improved.

[0067] It should be noted that, in order to ensure the signal quality of the audio stream, the downsampling rate may be preset to be no more than 50%. The specific selection is made according to the actual situation, and the embodiment of the present application does not limit it.

[0068] In some embodiments, in an audio stream, a window of a preset length is slid with a preset step size to obtain regional audio data corresponding to each sliding of the window; the audio amplitude change trend of the regional audio data is calculated to determine the change trend characteristics corresponding to the regional audio data; when the change trend characteristics corresponding to the regional audio data meet preset conditions, the regional audio data is determined as the target regional audio data.

[0069] That is to say, a sliding window method can be used to slide a window of preset length in the real-time audio stream with a preset step size, and the audio data contained in each sliding window is used as a region of audio data. The audio amplitude change trend of the region of audio data is calculated to determine the change trend characteristics corresponding to the region of audio data.

[0070] In some embodiments, the preset step length step=w, where w>1. The larger w is, the faster the calculation is. A suitable value can be taken according to the actual situation, such as w equal to 20, which does not affect the accuracy of the detection and can ensure the efficiency of the algorithm. The specific value of the preset step length is not limited in the embodiment of the present application.

[0071] In some embodiments, the preset length of the window is the sampling length of the abnormal sample audio data.

[0072] In some embodiments, the change trend characteristics corresponding to the regional audio data can be determined by performing first-order difference or second-order difference calculation on the audio amplitude data corresponding to the regional sampling points. As shown in formula (1), as follows:

[0073]

[0074] In formula (1), j represents the jth window that is slid to, and X irepresents the audio amplitude data corresponding to each regional sampling point i in the regional audio data corresponding to the jth window. In some embodiments, X i Can refer to the original audio amplitude data y at each regional sampling point in the regional audio data i The absolute value of X i＝ abs(y i ). diff represents the first-order difference calculation. By using formula (1), the sum of the first-order differences of the audio amplitude data at the sampling points in each region within the window range is calculated as the change trend feature corresponding to the regional audio data.

[0075] In some embodiments, different calculation methods need to be modified according to actual test scenarios and requirements, such as calculating second-order differences or other calculation methods, etc. The specific selection is made according to actual conditions and is not limited in the embodiments of the present application.

[0076] In some embodiments, the preset condition includes: among the preset number of change trend features obtained by sliding window detection, the change trend feature of the regional audio data is the smallest or largest n change trend features, and n is greater than or equal to 1. The value of the preset number can be selected according to actual conditions and is not limited in the embodiments of the present application.

[0077] That is to say, in an audio stream, the area audio data with the smallest change trend characteristics (such as the smallest sum of first-order differences) can be determined as the target area audio data; or, the area audio data with the largest change trend characteristics (such as the largest sum of first-order differences) can be determined as the target area audio data; or, multiple areas audio data with the smallest change trend characteristics can be determined as the target area audio data; or, multiple areas audio data with the largest change trend characteristics can be determined as the target area audio data. The specific selection is made according to the actual situation and is not limited in the embodiments of the present application.

[0078] It should be noted that, when the target area audio data includes multiple area audio data, the subsequent matching process is to calculate the target area audio features corresponding to each area audio data in the multiple area audio data, and match the target area audio features corresponding to each area audio data with the abnormal audio features in the preset feature library to determine the abnormal detection results corresponding to each area audio data.

[0079] S102: Perform at least one data statistic based on the target area audio data to determine the target area audio features corresponding to the target area audio data.

[0080] In S102, the target area audio data includes a certain number of sampling points and the audio amplitude corresponding to each sampling point. According to the audio amplitude corresponding to each sampling point in the target area audio data, at least one data statistic is performed to obtain at least one data statistic result; according to each data statistic result in the at least one data statistic result, an audio feature value is obtained, thereby obtaining at least one audio feature value corresponding to the target area audio data as the target area audio feature corresponding to the target area audio data.

[0081] In some embodiments, the at least one data statistic may include at least one of the following: a mean value calculation, a standard deviation calculation, and a first-order difference calculation.

[0082] Exemplarily, by performing an average value calculation on the audio amplitude data corresponding to each sampling point in the target area audio data, the audio amplitude average value corresponding to the target area audio data can be obtained as an audio feature value corresponding to the target area audio data; by performing a standard deviation calculation on the audio amplitude data corresponding to each sampling point in the target area audio data, the audio amplitude standard deviation corresponding to the target area audio data can be obtained as an audio feature value corresponding to the target area audio data; by performing a first-order difference calculation on the audio amplitude data corresponding to each sampling point in the target area audio data, the audio amplitude first-order difference corresponding to the target area audio data can be obtained as an audio feature value corresponding to the target area audio data.

[0083] It should be noted that, in actual applications, other data statistical algorithms can also be used to obtain the audio feature values ​​corresponding to the audio data of the target area. The specific selection is made according to the actual situation and is not limited in the embodiments of the present application.

[0084] S103: performing at least one of abnormal audio feature matching of a preset feature library and feature comparison of adjacent audio region data on the target area audio feature to determine an abnormal audio feature detection result of the target area audio feature.

[0085] In S103, by matching the target area audio features with the abnormal audio features recorded in the preset feature library, the abnormal detection result of the regional audio features can be determined. Alternatively, by performing a data feature comparison between the target area audio features and the features of the adjacent audio area data corresponding to the target area audio data, the abnormal detection result of the target area audio features can be determined. In other words, the abnormal detection result of the target area audio features can be determined by performing a detection method of matching the abnormal audio features of the preset feature library with the target area audio features alone, or performing a detection method of comparing the features of the adjacent audio area data alone.

[0086] In S103, the feature matching of the preset feature library and the feature comparison of the adjacent audio area data can also be combined to determine the abnormal detection result of the audio feature of the target area. When combining the two detection methods, the feature matching of the preset feature library and the feature comparison of the adjacent audio area data can be performed in any order or simultaneously according to the needs of the actual situation. The specific selection is based on the actual situation, and the embodiments of the present application are not limited.

[0087] In some embodiments, the abnormality detection result of the target area audio feature may include: whether the target area audio data is abnormal audio data. Alternatively, the abnormality detection result of the target area audio feature may include: whether the target area audio data is abnormal audio data, and the abnormality type corresponding to the target area audio data.

[0088] In an embodiment of the present application, for a detection method of feature matching of a preset feature library, the preset feature library includes abnormal audio features obtained by pre-statisticing at least one abnormal sample audio data. By matching the target area audio features in the audio stream with the abnormal audio features in the preset feature library, the degree of match between the target area audio features and the known abnormal audio features can be determined, and then it can be determined whether the target area audio features are abnormal audio data. In some embodiments, further, the preset feature library can also record the abnormal type corresponding to the abnormal audio feature. If the target area audio feature has a high degree of match with a certain abnormal audio feature in the preset feature library, it can be determined that the target area audio data is abnormal audio data, and the abnormal type of the target area audio data is determined based on the abnormal type corresponding to the abnormal audio feature.

[0089] In an embodiment of the present application, for the detection method of feature comparison of adjacent audio area data, by performing feature comparison between the audio features of the target area and the audio data of the audio area adjacent to the target area audio data, the difference between the audio features of the target area and the features of the audio data of the adjacent area can be determined, thereby detecting the change of audio data in the audio stream, and then determining whether the change of the audio data is abnormal.

[0090] It can be understood that by detecting the audio amplitude change trend of the audio stream, the target area audio data that may be abnormal in the audio stream is determined; at least one data statistic is performed based on the target area audio data to determine the target area audio features corresponding to the target area audio data; the target area audio features are matched with the abnormal audio features in the preset feature library, and / or, based on the target area audio features and the adjacent audio area data corresponding to the target audio area data, the abnormal detection result of the target area audio data is determined. In this way, the data features of the audio data are calculated in real time by means of data statistics and matched with the preset feature library, and / or, audio anomaly detection is achieved by comparing the features of the target area audio data with the features of the adjacent audio area data. The algorithm has low complexity and small amount of calculation, so that the audio stream can be continuously and automatically detected in real time without missing any audio data in the real-time audio stream, thereby improving the real-time and accuracy of audio detection.

[0091] In the embodiment of the present application, the preset feature library includes at least one abnormal audio feature corresponding to each abnormal type in at least one preset abnormal type; the at least one abnormal audio feature corresponding to each abnormal type is obtained by performing at least one data statistic on at least one abnormal sample audio data corresponding to the abnormal type. That is to say, by performing at least one data statistic on each abnormal sample audio data in at least one abnormal sample audio data corresponding to each abnormal type, the abnormal audio feature corresponding to each abnormal sample audio data can be obtained; and then the at least one abnormal audio feature corresponding to each abnormal type is determined according to the at least one abnormal sample audio data. Thus, the at least one abnormal audio feature corresponding to each abnormal type in at least one abnormal type can be recorded in the preset feature library, and used for matching with the target area audio feature of the target area audio data in the audio stream during audio stream detection to determine whether the target area audio data is abnormal. In this way, even for an abnormal type with a low probability of occurrence, as long as a small number of samples of the abnormal type are obtained, the abnormal audio feature corresponding to the abnormal type can be obtained in a statistical manner, thereby achieving more comprehensive abnormal detection.

[0092] In some embodiments, based on Figure 1 In the above S103, the process of performing at least one of abnormal audio feature matching of a preset feature library and adjacent audio region data feature comparison on the target area audio feature to determine the abnormal audio feature detection result of the target area audio feature can be as follows: Figure 2 As shown, it is achieved by executing S201, including: matching the audio features of the target area with the abnormal audio features in the preset feature library, and determining the abnormality detection results corresponding to the audio data of the target area.

[0093] That is, the detection method of feature matching of the preset feature library may include: matching the target area audio feature with the abnormal audio feature in the preset feature library to determine the abnormality detection result corresponding to the target area audio data. The process is described below.

[0094] In some embodiments, the target area audio feature includes: at least one audio feature value corresponding to at least one data statistical method. At least one audio feature value of the target area audio data is matched with at least one abnormal audio feature corresponding to each abnormal type in at least one abnormal type in the preset feature library. If the matching degree of at least one audio feature value and the abnormal audio feature corresponding to a certain abnormal type is high, exemplarily, the matching degree is greater than or equal to the preset matching threshold, it means that the target area audio data is very close to the audio feature of the abnormal type, so that the target area audio data in the audio stream can be determined to be abnormal audio data. In some embodiments, further, the abnormal type of the target area audio data can also be determined as the abnormal type corresponding to the abnormal audio feature matched therewith. On the contrary, if at least one audio feature value and at least one abnormal audio feature corresponding to each abnormal type in at least one abnormal type in the preset feature library are both low, exemplarily, the matching degree is less than the preset matching threshold, indicating that the possibility of the target area audio data being abnormal audio data is low, and it can be determined that the target area audio data in the audio stream is not abnormal audio data. Thus, the abnormal detection result of the target area audio data is obtained.

[0095] In some embodiments, the audio feature values ​​with the same data statistics method can be matched with the abnormal audio features. Exemplarily, the average value of the audio data in the target area is matched with the abnormal audio feature obtained by calculating the average value of at least one abnormal sample audio data; the standard deviation of the audio data in the target area is matched with the abnormal audio feature obtained by calculating the standard deviation of at least one abnormal sample audio data; the first-order difference of the audio data in the target area is matched with the abnormal audio feature obtained by calculating the first-order difference of at least one abnormal sample audio data.

[0096] It can be understood that by matching the audio features of the target area with the preset feature library, compared with the current model reasoning, the computational workload is small, it can be better applied to the scenario of real-time audio detection, and the accuracy is high, thereby improving the real-time and accuracy of audio stream detection.

[0097] In some embodiments, based on Figure 1 In S103, the process of performing at least one of abnormal audio feature matching of a preset feature library and adjacent audio region data feature comparison on the target area audio feature to determine the abnormal audio feature detection result of the target area audio feature can be as follows: Figure 3As shown, it is achieved by executing S301, including: performing data feature comparison according to the audio features of the target area and the adjacent audio area data corresponding to the target audio area data, and determining the abnormality detection result corresponding to the audio data of the target area.

[0098] That is, the detection method of the feature comparison of the adjacent audio region data may include: matching the target region audio feature with the abnormal audio feature in the preset feature library to determine the abnormality detection result corresponding to the target region audio data. The process is described below.

[0099] In some embodiments, the adjacent audio area data corresponding to the target audio area data may include: audio area data before the target audio area data and audio area data after the target audio area data. By performing data statistics on the audio area data before the target audio area data and the audio area data after the target audio area data, the regional audio features corresponding to the audio area data before the target audio area data and the regional audio features corresponding to the audio area data after the target audio area data can be determined. The target area audio features are compared with the regional audio features corresponding to the audio area data before the target audio area data and the regional audio features corresponding to the audio area data after the target audio area data to determine the abnormality detection result corresponding to the target area audio data.

[0100] Exemplarily, when the difference between the target area audio feature and the area audio feature corresponding to the audio area data before the target audio area data is greater than a preset threshold, and the difference between the target area audio feature and the area audio feature corresponding to the audio area data after the target audio area data is greater than the preset threshold, it means that the target area audio data is significantly different from the audio data before and after it, and the target area audio data can be determined to be abnormal audio data.

[0101] In some embodiments, an average value can be calculated based on the audio amplitude data corresponding to the sampling points in the target area audio data to determine the average value corresponding to the audio amplitude data in the target area audio data as the target area audio feature. The area audio data corresponding to the previous window of the target area audio data is used as the first area audio data, and the area audio data corresponding to the next window of the target area audio data is used as the second area audio data. The process of comparing data features based on the target area audio feature and the adjacent audio area data corresponding to the target audio area data in S301 to determine the abnormality detection result corresponding to the target area audio data can be as follows: Figure 4 As shown below:

[0102] S3011. Obtain first area audio data and second area audio data.

[0103] S3012. Calculate a first average value corresponding to the audio amplitude data in the audio data of the first region and a second average value corresponding to the audio amplitude data in the audio data of the second region.

[0104] S3013. Determine an abnormality detection result corresponding to the target area audio data according to the first average value, the second average value, and the average value corresponding to the audio amplitude data in the target area audio data.

[0105] In some embodiments, when the ratio of the first average value to the audio amplitude data in the target area audio data is greater than a preset ratio threshold, and the ratio of the second average value to the audio amplitude data in the target area audio data is greater than a preset ratio threshold, the target area audio data is determined to be abnormal audio data.

[0106] For example, for Figure 5 The audio stream shown in the figure can determine that the audio data in the target area is the audio data in the dotted box in the figure, which is recorded as M(X). Get the sampling intervals with the same length before and after this area, such as Figure 6 As shown in the solid line box in the figure, the regional audio data of the previous sampling interval of M(X) is taken as the first regional audio data, denoted as M f (X), and the regional audio data of the next sampling interval after M(X) is taken as the second regional audio data, denoted as M e (X), according to formula (2) to formula (4), calculate the first average value Avg m 、The second average value Avg f And the average value Avg corresponding to the audio amplitude data in the target area audio data e ,as follows:

[0107] Avg m =average(M(X)) Formula (2)

[0108] Avg f =average(M f (X)) Formula (3)

[0109] Avg e =average(M e (X)) Formula (4)

[0110] If Avg f / Avg m >a, and Avg e / Avg m >a, the target area audio data is abnormal audio data. Exemplarily, the preset ratio threshold a may be 2. Other values ​​may also be taken according to actual conditions, which are not limited in the present embodiment of the application.

[0111] It can be understood that by detecting the audio amplitude change trend of the audio stream and determining the target area audio data that may be abnormal in the audio stream, the algorithm has low complexity and small amount of calculation, so that the audio stream can be continuously and automatically detected in real time without missing any audio data in the audio stream, thereby improving the real-time and accuracy of audio detection.

[0112] In some embodiments, based on Figure 1 In the above S103, the process of performing at least one of abnormal audio feature matching of a preset feature library and adjacent audio region data feature comparison on the target area audio feature to determine the abnormal audio feature detection result of the target area audio feature can be as follows: Figure 7 As shown, it is implemented by executing S401-S402, including:

[0113] S401: Match the target area audio features with abnormal audio features in a preset feature library to determine a first abnormality detection result corresponding to the target area audio data.

[0114] S402. When the first abnormality detection result indicates that the target area audio feature is not abnormal audio data, data feature comparison is performed based on the target area audio feature and adjacent audio area data corresponding to the target audio area data to determine the abnormality detection result corresponding to the target area audio data.

[0115] In some embodiments, when the first abnormality detection result represents that the audio feature of the target area is abnormal audio data, the first abnormality detection result can be directly determined as the abnormality detection result corresponding to the audio data of the target area.

[0116] That is to say, when combining the feature matching of the preset feature library and the feature comparison of the adjacent audio area data, the target area audio feature can be first matched with the abnormal audio feature in the preset feature library, and the abnormal detection result determined by the detection method of the feature matching method is used as the first abnormal detection result. In the case where the first abnormal detection result indicates that the target area audio feature is not abnormal audio data, further detection is performed according to the detection method of the feature comparison of the adjacent audio area data, and the abnormal detection result corresponding to the target area audio data is determined by performing feature comparison based on the target area audio feature and the adjacent audio area data corresponding to the target audio area data.

[0117] Here, the process of matching the target area audio features with the abnormal audio features in the preset feature library is consistent with the process description of feature matching in S201 above; the process of comparing data features based on the target area audio features and the adjacent audio area data corresponding to the target audio area data is consistent with the process description of feature comparison in S301 above, and will not be repeated here.

[0118] It can be understood that by first performing preliminary detection by means of feature library matching, and then further detecting by means of feature comparison of adjacent audio area data when no abnormality is found in the preliminary detection, the abnormality detection result corresponding to the final target area audio data is determined, thereby improving the accuracy of audio detection.

[0119] In some embodiments, based on Figure 1 In the above S103, the process of performing at least one of abnormal audio feature matching of a preset feature library and adjacent audio region data feature comparison on the target area audio feature to determine the abnormal audio feature detection result of the target area audio feature can be as follows: Figure 8 As shown, it is implemented by executing S501-S502, including:

[0120] S501: Perform data feature comparison based on the target area audio feature and the adjacent audio area data corresponding to the target area audio data to determine a second abnormality detection result corresponding to the target area audio data.

[0121] S502: When the second abnormality detection result indicates that the target area audio feature is not abnormal audio data, the target area audio feature is matched with the abnormal audio feature in a preset feature library to determine the abnormality detection result corresponding to the target area audio data.

[0122] In some embodiments, when the second abnormality detection result represents that the audio feature of the target area is abnormal audio data, the second abnormality detection result can be directly determined as the abnormality detection result corresponding to the audio data of the target area.

[0123] That is to say, when combining the feature matching of the preset feature library and the feature comparison of the adjacent audio area data, the data feature comparison can be first performed based on the target area audio feature and the adjacent audio area data corresponding to the target audio area data, and the abnormal detection result determined by the detection method of the feature comparison of the adjacent audio area data is used as the second abnormal detection result. In the case where the second abnormal detection result indicates that the target area audio feature is not abnormal audio data, further detection is performed according to the detection method of the feature matching of the preset feature library, and the target area audio feature is matched with the abnormal audio feature in the preset feature library to determine the abnormal detection result corresponding to the target area audio data.

[0124] It can be understood that by first performing a preliminary detection by comparing the features of adjacent audio area data, and then further detecting by feature library matching when no abnormality is found in the preliminary detection, the abnormality detection result corresponding to the final target area audio data is determined, thereby improving the accuracy of audio detection.

[0125] In some embodiments, based on Figure 1 In the above S103, the process of performing at least one of abnormal audio feature matching of a preset feature library and adjacent audio region data feature comparison on the target area audio feature to determine the abnormal audio feature detection result of the target area audio feature can be as follows: Fig. 9 As shown, it is implemented by executing S601-S602, including:

[0126] S601. Match the target area audio features with the abnormal audio features in the preset feature library to determine a first abnormality detection result corresponding to the target area audio data, and perform data feature comparison based on the target area audio features and the adjacent audio area data corresponding to the target audio area data to determine a second abnormality detection result corresponding to the target area audio data.

[0127] S602: Determine an abnormality detection result of the target audio feature according to the first abnormality detection result and the second abnormality detection result.

[0128] That is to say, the feature matching of the preset feature library and the feature comparison of the adjacent audio region data can be performed synchronously or simultaneously to obtain the first abnormality detection result and the second abnormality detection result. Then, according to the first abnormality detection result and the second abnormality detection result, the final abnormality detection result of the target audio feature is determined.

[0129] In some embodiments, when both the first abnormality detection result and the second abnormality detection result indicate that the target area audio feature is not abnormal audio data, it is determined that the target area audio feature is not abnormal audio data. When at least one of the first abnormality detection result and the second abnormality detection result indicates that the target area audio data is abnormal audio data, it is determined that the target area audio data is abnormal audio data. In other words, as long as one of the first abnormality detection result and the second abnormality detection result is abnormal, the target area audio data is determined to be abnormal audio data.

[0130] It can be understood that by simultaneously performing feature library matching and feature comparison of adjacent audio area data for detection, the anomaly detection result corresponding to the final target area audio data is determined by combining the anomaly detection results of the two, thereby improving the accuracy of audio detection.

[0131] In some embodiments, before using the preset feature library for feature matching detection, Fig.10 The process of S001-S004 shown in the figure establishes a preset feature library as follows:

[0132] S001. Obtain at least one abnormal sample audio data corresponding to each abnormal type in at least one abnormal type.

[0133] In S001, for the acquisition stage of abnormal sample audio data, when an audio anomaly is found or reproduced, the abnormal sample audio data can be acquired through tool recording, audio log, etc. For the algorithm verification or actual anomaly detection stage in S101, the audio stream can be acquired in real time through the sound card for real-time detection. The embodiment of the present application realizes small-scale detection, that is, for each abnormal type, only one or a small number of abnormal sample audio data can be acquired for feature extraction.

[0134] S002. For each abnormal sample audio data in the at least one abnormal sample audio data, perform at least one data statistic according to the audio amplitude data in each abnormal sample audio data to determine at least one characteristic statistic value corresponding to each abnormal sample audio data.

[0135] S003. Determine an abnormal audio feature corresponding to each abnormal sample audio data according to at least one feature statistic value and at least one of the audio amplitude data.

[0136] In the embodiment of the present application, each abnormal sample audio data includes audio amplitude data yi of sampling point length ki. Each abnormal sample audio data may also include a pre-marked abnormal type. For each abnormal sample audio data, at least one characteristic statistical value and at least one of the audio amplitude data of the abnormal sample audio data may be determined as an abnormal audio feature corresponding to the abnormal sample audio data.

[0137] In some embodiments, the above Figure 5 and Figure 6 This represents a typical type of stuttering phenomenon, where the audio amplitude in the frame is significantly lower than the audio amplitude before and after it. Figure 5 and Figure 6 The abnormal audio data of the first abnormal type in the audio stream shown can be detected by determining the abnormal detection result of the target area audio feature through a detection method of comparing the target area audio feature with the feature of the adjacent audio area data corresponding to the target area audio data, thereby detecting the abnormal audio data of the first abnormal type.

[0138] In some embodiments, Fig.11 Represents a special type of irregular lag, with some extreme values ​​appearing. Fig.11The abnormal audio data of the second abnormal type in the audio stream shown can be detected by detecting the abnormal audio data of the second abnormal type through a preset feature library. Here, at least one abnormal sample audio data corresponding to the abnormal type is collected in advance, and the abnormal audio features corresponding to each abnormal sample audio data are determined by the method in S002-S003, and recorded in the preset feature library for subsequent audio detection of this abnormal type. Among them, the process of determining the abnormal audio features corresponding to each abnormal sample audio data according to at least one characteristic statistical value and at least one of the audio amplitude data in the above S003 may include: determining at least one characteristic statistical value range according to at least one characteristic statistical value and a preset fluctuation amount; determining the abnormal audio features corresponding to each abnormal sample audio data according to at least one characteristic statistical value range and at least one of the audio amplitude data.

[0139] For example, as shown in formula (5) to formula (9), it is as follows:

[0140] mean i =[mean(i)*(1-p),mean(i)*(1+p)] Formula (5)

[0141] std i =[std(i)*(1-p),std(i)*(1+p)] Formula (6)

[0142] diff i =[diff(i)*(1-p),diff(i)*(1+p)] Formula (7)

[0143] fi=(k i ,mean i , std i , diff i ) Formula (8)

[0144] F2 s =(f1, f2,...,f i ) Formula (9)

[0145] Where fi is the abnormal audio feature corresponding to the i-th abnormal sample audio data; F2 s is at least one abnormal audio feature corresponding to the abnormal type, that is, it includes the abnormal audio feature corresponding to each abnormal sample audio data in the at least one abnormal sample audio data corresponding to the abnormal type. i Indicates the average value of the audio amplitude data corresponding to the i-th abnormal sample audio data, which is used to reflect the center position of the abnormal interval data value as a whole; std iIndicates the standard deviation of the audio amplitude data corresponding to the i-th abnormal sample audio data, which is used to reflect the degree of deviation between the value of the abnormal interval and the average value; diff i Indicates the first-order difference of the audio amplitude data corresponding to the i-th abnormal sample audio data, which is used to reflect the trend of the interval value changing over time. i Represents the audio amplitude data corresponding to each sampling point in the i-th abnormal sample audio data. p is a preset fluctuation amount. Exemplarily, the value range of p can be 0<p<1. According to actual tests, it can be found that the characteristic value of the same abnormality will not be exactly the same each time it occurs, but will fluctuate within a small range. Therefore, the fluctuation parameter is set to p. Exemplarily, p can be 0.1, which is selected according to the actual situation and is not limited in the embodiments of the present application.

[0146] In some embodiments, when matching the target area audio feature in the preset feature library, at least one audio feature value in the target area audio feature can be matched with at least one feature statistical value range contained in each abnormal audio feature in the preset feature library to determine whether the audio feature value falls into the corresponding feature statistical value range; according to the situation where at least one audio feature value falls into the corresponding feature statistical value range, such as the number of falls, or the preset weight corresponding to the feature statistical value range, etc., the matching degree of the target area audio feature and the corresponding abnormal audio feature is determined. Exemplarily, if the number of at least one audio feature value falling into the corresponding feature statistical value range of a certain abnormal audio feature is greater than the preset number threshold, it is determined that the matching degree of the target area audio feature and the abnormal audio feature is greater than the preset matching degree threshold; and / or, when a certain audio feature value falls into the feature statistical value range with a higher weight, it is determined that the matching degree of the target area audio feature and the abnormal audio feature is greater than the preset matching degree threshold. Then, it can be determined that the target area audio data is abnormal audio data of the abnormal type corresponding to the abnormal audio feature. In actual applications, the matching strategy is selected according to the actual situation. The above is only an example, and the embodiments of the present application are not specifically limited.

[0147] S004. Determine at least one abnormal audio feature corresponding to each abnormal type according to the abnormal audio feature corresponding to each abnormal sample audio data in at least one abnormal sample audio data, thereby determining a preset feature library.

[0148] In S004, according to the abnormal audio features corresponding to each abnormal sample audio data, the abnormal audio features corresponding to each abnormal sample audio data in at least one abnormal sample audio data can be obtained as at least one abnormal audio feature corresponding to each abnormal type. For each abnormal type, at least one abnormal audio feature corresponding to each abnormal sample audio data can be recorded in the preset feature library under the abnormal type as the abnormal audio feature corresponding to the abnormal type. Alternatively, data fusion processing can be performed according to at least one abnormal audio feature corresponding to the abnormal type to obtain a fused feature as the abnormal audio feature corresponding to the abnormal type. The specific selection is made according to the actual situation, and the embodiment of the present application is not limited thereto.

[0149] It should be noted that the above-mentioned first abnormal type and second abnormal type are merely examples of abnormal audio types. The abnormal audio detection method in the embodiment of the present application can also be extended to detect other abnormal audio features, which is not specifically limited here.

[0150] It can be understood that for each abnormal type of audio abnormality, the preset feature library of the embodiment of the present application only needs a small amount of abnormal sample audio data to perform data statistics to obtain the abnormal audio features corresponding to the abnormal type, and record them in the preset feature library to match at least one audio feature value, thereby realizing the detection and identification of more types of audio abnormalities, further improving the accuracy of audio detection.

[0151] In some embodiments, at least one anomaly detection result obtained by performing anomaly detection on at least one audio stream and at least one annotation information corresponding to the at least one anomaly detection result can also be obtained; and the abnormal audio features in the preset feature library can be updated according to the at least one annotation information corresponding to the at least one anomaly detection result.

[0152] Among them, at least one annotation information can be obtained by manually verifying and annotating at least one anomaly detection result. In this way, through actual audio anomaly detection or the accumulation of audio data during the test process, it can help to reproduce the anomaly and find the cause of the anomaly; on the other hand, it can continuously verify the effectiveness of the algorithm and discover new anomaly features, enrich the preset feature library, and continuously optimize the algorithm.

[0153] It is understandable that the embodiments of the present application realize the automatic detection of audio streams, such as the audio streams collected by the vehicle voice system, which can avoid a large amount of manual detection workload, improve detection efficiency, and save costs. The embodiments of the present application can extract abnormal features through automatic detection based on a small number of abnormal data samples, and can accurately detect audio anomalies such as freezes, mutations (sudden changes), short silences, and regular noises; the detection algorithm has a small amount of calculation, which ensures the speed of real-time detection, and can continuously detect audio data for a long time without missing any audio data. In addition, a large amount of detection data can be accumulated, which is conducive to further optimizing and upgrading the performance of the voice system, and providing data support for improving the intelligence and humanization of the overall system. Thereby, the accuracy and real-time performance of audio anomaly detection are improved.

[0154] Based on the above abnormal audio detection method provided in the embodiment of the present application, the embodiment of the present application provides a method as follows: Fig.12 The abnormal audio detection system shown is applied to abnormal audio detection in vehicle systems. In the data acquisition link, when audio abnormalities are found or reproduced on the vehicle, abnormal sample audio data can be obtained through manual collection, such as tool recording, audio logs, etc. When performing real-time abnormal audio detection on the vehicle, the real-time audio stream is obtained through the sound card. In the algorithm processing link, at least one data statistic is performed on the abnormal sample audio data to obtain abnormal audio features corresponding to its abnormal type, which are stored in a preset feature library. For the real-time audio stream, the audio data in the target area is determined by detecting the audio amplitude change area, and at least one data statistic is performed based on the audio data in the target area to determine the audio features of the target area, and then the audio features of the target area are matched in the preset feature library to obtain the abnormal detection results corresponding to the audio data in the target area. Figure 7 As shown, in the test mode, the same abnormal audio detection process can be performed on the real-time audio stream to obtain the test result, such as determining the target area audio data, determining the target area audio features, and matching with the preset feature library to obtain the abnormal detection result. The test result includes the abnormal detection result corresponding to the target area audio data in the real-time audio stream, that is, the abnormal type corresponding to the abnormal audio data in the real-time audio stream. Storing the test results in the preset feature library can further enrich the abnormal types in the preset feature library and improve the detection rate of abnormal audio.

[0155] Exemplarily, the present application embodiment provides an abnormal audio detection method applied to actual scenarios, such as Fig.13As shown, it includes: downsampling the mono channel in the acquired real-time audio stream; using a sliding window, performing first-order difference calculation on the regional audio data in each window, and determining the target region audio data according to the minimum value of the first-order difference, that is, the regional audio data in the window with the minimum value of the first-order difference is used as the target region audio data. Perform at least one data feature statistics on the target region audio data to determine the target region audio feature corresponding to the target region audio data. By performing feature matching on the target region audio feature and the abnormal audio feature in the preset feature library, the first abnormal detection result corresponding to the target region audio data is determined. Among them, the abnormal audio feature in the preset feature library is obtained by performing feature statistics on the abnormal sample audio data, that is, at least one data statistic. The target region audio feature includes the average value corresponding to the audio amplitude data in the target region audio data; while matching the features, the audio amplitude data in the regional audio data of the front and rear windows of the target region audio data is averaged to determine the first average value and the second average value. By performing a difference threshold judgment on the average value corresponding to the audio amplitude data in the target region audio data, the first average value and the second average value, the second abnormal detection result corresponding to the target region audio data is determined. According to the first abnormality detection result and / or the second abnormality detection result, determine the abnormality detection result of the target audio feature. If the target area audio data is not abnormal audio data, end the detection of the real-time audio stream, and can restart the detection of the next real-time audio stream as needed. If the target area audio data is abnormal audio data, save the abnormal file and log corresponding to the abnormal audio data. Exemplarily, the abnormal file can record the audio stream corresponding to the abnormal audio data, the abnormal type and other information, and the log can record the occurrence time of the abnormal audio data, the detection time and the detection process and other information.

[0156] It can be understood that the embodiments of the present application implement an automated audio anomaly detection process, and anomalies can be detected on real-time audio streams through detection algorithms; by setting appropriate thresholds and performing threshold judgments on possible abnormal areas and front and back areas, anomalies can be detected more accurately; or, feature matching methods are used to detect anomalies, thereby meeting the needs of real-time audio detection, and can be used in large-scale detection scenarios without the need for human intervention, saving costs and improving detection efficiency. Further, the detection rate of abnormal audio is improved, and the detection accuracy is high.

[0157] The present application embodiment provides an abnormal audio detection device 1, such as Fig.14 As shown, including:

[0158] A detection module 11, configured to determine target area audio data in the audio stream by detecting a change trend of audio amplitude of the audio stream;

[0159] A statistical module 12, configured to perform at least one data statistic based on the target area audio data to determine a target area audio feature corresponding to the target area audio data;

[0160] The determination module 13 is used to perform at least one of abnormal audio feature matching of a preset feature library and feature comparison of adjacent audio area data on the target area audio feature to determine an abnormality detection result of the target area audio feature.

[0161] In some embodiments, the determination module 13 is also used to match the target area audio features with abnormal audio features in a preset feature library to determine the abnormal detection results corresponding to the target area audio data; the abnormal audio features are obtained by performing at least one data statistic on at least one abnormal sample audio data corresponding to at least one abnormal type.

[0162] In some embodiments, the determination module 13 is further used to perform data feature comparison based on the target area audio features and adjacent audio area data corresponding to the target area audio data, to determine an abnormality detection result corresponding to the target area audio data.

[0163] In some embodiments, the determination module 13 is also used to match the target area audio features with abnormal audio features in a preset feature library to determine a first abnormality detection result corresponding to the target area audio data; when the first abnormality detection result indicates that the target area audio features are not abnormal audio data, data features are compared based on the target area audio features and adjacent audio area data corresponding to the target audio area data to determine the abnormality detection result corresponding to the target area audio data.

[0164] In some embodiments, the determination module 13 is also used to perform data feature comparison based on the target area audio feature and the adjacent audio area data corresponding to the target audio area data, to determine a second abnormality detection result corresponding to the target area audio data; when the second abnormality detection result indicates that the target area audio feature is not abnormal audio data, the target area audio feature is matched with the abnormal audio feature in a preset feature library to determine the abnormality detection result corresponding to the target area audio data.

[0165] In some embodiments, the determination module 13 is also used to match the target area audio features with the abnormal audio features in a preset feature library to determine a first abnormality detection result corresponding to the target area audio data, and to perform data feature comparison based on the target area audio features and the adjacent audio area data corresponding to the target audio area data to determine a second abnormality detection result corresponding to the target area audio data; based on the first abnormality detection result and / or the second abnormality detection result, determine the abnormality detection result of the target audio feature.

[0166] In some embodiments, the determination module 13 is also used to determine that the target area audio data is not abnormal audio data when both the first abnormality detection result and the second abnormality detection result indicate that the target area audio data is not abnormal audio data; and to determine that the target area audio data is abnormal audio data when at least one of the first abnormality detection result and the second abnormality detection result indicates that the target area audio data is abnormal audio data.

[0167] In some embodiments, the abnormal audio detection device 1 also includes a construction module, which is used to obtain at least one abnormal sample audio data corresponding to each abnormal type in at least one abnormal type; for each abnormal sample audio data in the at least one abnormal sample audio data, at least one data statistics is performed according to the audio amplitude data in each abnormal sample audio data to determine at least one feature statistic value corresponding to each abnormal sample audio data; according to the at least one feature statistic value and at least one of the audio amplitude data, the abnormal audio feature corresponding to each abnormal sample audio data is determined; according to the abnormal audio feature corresponding to each abnormal sample audio data in the at least one abnormal sample audio data, at least one abnormal audio feature corresponding to each abnormal type is determined, thereby determining the preset feature library.

[0168] In some embodiments, the construction module is further used to determine at least one characteristic statistical value range based on the at least one characteristic statistical value and the preset fluctuation amount; and determine the abnormal audio feature corresponding to each abnormal sample audio data based on the at least one characteristic statistical value range and at least one of the audio amplitude data.

[0169] In some embodiments, the at least one data statistic includes at least one of the following:

[0170] Mean, standard deviation, and first-order difference calculations.

[0171] In some embodiments, the detection module 11 is further used to slide a window of a preset length in the audio stream with a preset step size to obtain regional audio data corresponding to each sliding of the window; calculate the audio amplitude change trend of the regional audio data to determine the change trend characteristics corresponding to the regional audio data; and determine the regional audio data as the target regional audio data when the change trend characteristics corresponding to the regional audio data meet preset conditions.

[0172] In some embodiments, the regional audio data includes: audio amplitude data corresponding to the regional sampling points; the detection module 11 is also used to determine the change trend characteristics corresponding to the regional audio data by performing first-order difference or second-order difference calculation on the audio amplitude data corresponding to the regional sampling points.

[0173] In some embodiments, the preset conditions include:

[0174] Among a preset number of change trend features obtained by sliding window detection, the change trend features of the regional audio data are the smallest or largest n change trend features, where n is greater than or equal to 1.

[0175] In some embodiments, the target area audio features include: the average value corresponding to the audio amplitude data in the target area audio data; the adjacent audio area data corresponding to the target audio area data include: the first area audio data corresponding to the previous window of the target area audio data, and the second area audio data corresponding to the next window of the target area audio data; the determination module 13 is also used to obtain the first area audio data and the second area audio data; calculate the first average value corresponding to the audio amplitude data in the first area audio data and the second average value corresponding to the audio amplitude data in the second area audio data; determine the abnormality detection result corresponding to the target area audio data according to the first average value, the second average value and the average value corresponding to the audio amplitude data in the target area audio data.

[0176] In some embodiments, the determination module 13 is further used to determine that the target area audio data is abnormal audio data when the ratio of the first average value to the average value corresponding to the audio amplitude data in the target area audio data is greater than a preset ratio threshold, and the ratio of the second average value to the average value corresponding to the audio amplitude data in the target area audio data is greater than the preset ratio threshold.

[0177] In some embodiments, the detection module 11 is further used to obtain a mono audio stream in the original audio stream; and downsample the mono audio stream to determine the audio stream.

[0178] In some embodiments, the abnormal audio detection device 1 also includes an update module, which is used to obtain at least one abnormal detection result obtained by performing abnormal detection on at least one audio stream, and at least one annotation information corresponding to the at least one abnormal detection result; according to the at least one annotation information corresponding to the at least one abnormal detection result, the abnormal audio features in the preset feature library are updated.

[0179] It should be noted that the description of the above device embodiment is similar to the description of the above method embodiment, and has similar beneficial effects as the method embodiment. For technical details not disclosed in the device embodiment of the present invention, please refer to the description of the method embodiment of the present invention for understanding.

[0180] The present application also provides an electronic device, Fig.15 This is an optional structural diagram of the electronic device 3 provided in the embodiment of the present application. Fig.15 As shown, the electronic device 3 includes: a memory 32 and a processor 33. The memory 32 and the processor 33 are connected via a communication bus 34; the memory 32 is used to store executable instructions; the processor 33 is used to implement the abnormal audio detection method provided in the embodiment of the present application when executing the executable instructions stored in the memory 32.

[0181] An embodiment of the present application provides a computer-readable storage medium storing executable instructions, wherein executable instructions are stored. When the executable instructions are executed by the above-mentioned processor, the above-mentioned processor will execute the abnormal audio detection method provided in the embodiment of the present application.

[0182] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface storage, optical disk, or CD-ROM; or it may be various devices including one or any combination of the above memories.

[0183] In some embodiments, executable instructions may be in the form of a program, software, software module, script or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine or other unit suitable for use in a computing environment.

[0184] As examples, executable instructions may, but need not, correspond to a file in a file system, may be stored as part of a file storing other programs or data, such as one or more scripts in a Hyper Text Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files storing one or more modules, subroutines, or code portions). As examples, executable instructions may be deployed to be executed on one computing device, or on multiple computing devices located at one location, or on multiple computing devices distributed at multiple locations and interconnected by a communication network.

[0185] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of hardware embodiments, software embodiments, or embodiments in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) that contain computer-usable program code.

[0186] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0187] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0188] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0189] The above is only a preferred embodiment of the present application and is not intended to limit the protection scope of the present application. Any modifications, equivalent substitutions and improvements made within the spirit and scope of the present application are included in the protection scope of the present application.

Claims

1. A method for detecting abnormal audio, characterized in that: The method comprises: Determine the target area audio data in the audio stream by detecting the audio amplitude change trend of the audio stream; Perform at least one data statistic based on the target area audio data to determine the target area audio features corresponding to the target area audio data; The target area audio feature is subjected to at least one of abnormal audio feature matching in a preset feature library and feature comparison of adjacent audio area data to determine an abnormality detection result of the target area audio feature.

2. The method according to claim 1, characterized in that The performing at least one of abnormal audio feature matching of a preset feature library and feature comparison of adjacent audio region data on the target area audio feature to determine the abnormal audio feature detection result of the target area audio feature includes: Matching the target area audio feature with the abnormal audio feature in the preset feature library to determine the abnormality detection result corresponding to the target area audio data; the abnormal audio feature is obtained by performing at least one data statistic on at least one abnormal sample audio data corresponding to at least one abnormal type; Alternatively, data feature comparison is performed based on the target area audio feature and adjacent audio area data corresponding to the target area audio data to determine the abnormality detection result corresponding to the target area audio data.

3. The method according to claim 1, characterized in that The performing at least one of abnormal audio feature matching of a preset feature library and feature comparison of adjacent audio region data on the target area audio feature to determine the abnormal audio feature detection result of the target area audio feature includes: Matching the target area audio feature with the abnormal audio feature in a preset feature library to determine a first abnormality detection result corresponding to the target area audio data; When the first abnormality detection result indicates that the target area audio feature is not abnormal audio data, a data feature comparison is performed based on the target area audio feature and adjacent audio area data corresponding to the target audio area data to determine the abnormality detection result corresponding to the target area audio data.

4. The method according to claim 1, characterized in that: The performing at least one of abnormal audio feature matching of a preset feature library and feature comparison of adjacent audio region data on the target area audio feature to determine the abnormal audio feature detection result of the target area audio feature includes: Performing data feature comparison based on the target area audio feature and adjacent audio area data corresponding to the target area audio data, and determining a second abnormality detection result corresponding to the target area audio data; When the second abnormality detection result indicates that the target area audio feature is not abnormal audio data, the target area audio feature is matched with abnormal audio features in a preset feature library to determine the abnormality detection result corresponding to the target area audio data.

5. The method according to claim 1, characterized in that The performing at least one of abnormal audio feature matching of a preset feature library and feature comparison of adjacent audio region data on the target area audio feature to determine the abnormal audio feature detection result of the target area audio feature includes: Matching the target area audio feature with the abnormal audio feature in the preset feature library to determine a first abnormality detection result corresponding to the target area audio data, and performing data feature comparison based on the target area audio feature and adjacent audio area data corresponding to the target audio area data to determine a second abnormality detection result corresponding to the target area audio data; An abnormality detection result of the target audio feature is determined according to the first abnormality detection result and / or the second abnormality detection result.

6. The method according to any one of claims 1 to 5, characterized in that: The method further comprises: Obtain at least one abnormal sample audio data corresponding to each abnormal type of at least one abnormal type; For each abnormal sample audio data in the at least one abnormal sample audio data, performing at least one data statistic according to the audio amplitude data in each abnormal sample audio data, and determining at least one characteristic statistic value corresponding to each abnormal sample audio data; Determine, according to the at least one characteristic statistical value and at least one of the audio amplitude data, an abnormal audio feature corresponding to each abnormal sample audio data; According to the abnormal audio feature corresponding to each abnormal sample audio data in the at least one abnormal sample audio data, at least one abnormal audio feature corresponding to each abnormal type is determined, thereby determining the preset feature library.

7. The method according to any one of claims 1 to 5, characterized in that: The step of determining the target area audio data in the audio stream by detecting the audio amplitude change trend of the audio stream comprises: In the audio stream, sliding a window of a preset length with a preset step length, and obtaining regional audio data corresponding to each sliding of the window; Calculating the audio amplitude change trend of the regional audio data to determine the change trend characteristics corresponding to the regional audio data; When the change trend characteristics corresponding to the regional audio data meet a preset condition, the regional audio data is determined as the target regional audio data.

8. The method according to claim 7, characterized in that The regional audio data includes: audio amplitude data corresponding to the regional sampling points; the audio amplitude change trend calculation of the regional audio data to determine the change trend characteristics corresponding to the regional audio data includes: By performing first-order difference or second-order difference calculation on the audio amplitude data corresponding to the sampling points in the area, the change trend characteristics corresponding to the audio data in the area are determined.

9. The method according to claim 7, characterized in that: The preset conditions include: Among a preset number of change trend features obtained by sliding window detection, the change trend features of the regional audio data are the smallest or largest n change trend features, where n is greater than or equal to 1.

10. The method according to any one of claims 2 to 5, characterized in that: The target area audio feature includes: an average value corresponding to the audio amplitude data in the target area audio data; the adjacent audio area data corresponding to the target area audio data includes: first area audio data corresponding to a previous window of the target area audio data, and second area audio data corresponding to a subsequent window of the target area audio data; performing data feature comparison based on the target area audio feature and the adjacent audio area data corresponding to the target area audio data to determine the abnormality detection result corresponding to the target area audio data includes: Acquire the first area audio data and the second area audio data; Calculate a first average value corresponding to the audio amplitude data in the first area audio data and a second average value corresponding to the audio amplitude data in the second area audio data; An abnormality detection result corresponding to the target area audio data is determined according to the first average value, the second average value, and an average value corresponding to the audio amplitude data in the target area audio data.

11. The method according to claim 10, characterized in that The determining, according to the first average value, the second average value, and the average value corresponding to the audio amplitude data in the audio data of the target area, an abnormality detection result corresponding to the audio data of the target area includes: When the ratio of the first average value to the average value corresponding to the audio amplitude data in the target area audio data is greater than a preset ratio threshold, and the ratio of the second average value to the average value corresponding to the audio amplitude data in the target area audio data is greater than the preset ratio threshold, the target area audio data is determined to be abnormal audio data.

12. An abnormal audio detection device, characterized in that: The device comprises: A detection module, configured to determine target area audio data in the audio stream by detecting a change trend of audio amplitude of the audio stream; A statistical module, configured to perform at least one data statistic based on the target area audio data to determine a target area audio feature corresponding to the target area audio data; The determination module is used to perform at least one of abnormal audio feature matching of a preset feature library and feature comparison of adjacent audio area data on the target area audio feature to determine the abnormality detection result of the target area audio feature.

13. An electronic device, characterized in that: The electronic device comprises: Memory for storing computer executable instructions or computer programs; A processor, configured to implement the method according to any one of claims 1 to 11 when executing computer executable instructions or computer programs stored in the memory.

14. A computer-readable storage medium storing computer-executable instructions or a computer program, characterized in that: When the computer executable instructions or computer program are executed by a processor, the method according to any one of claims 1 to 11 is implemented.

15. A computer program product comprising computer executable instructions or a computer program, characterized in that When the computer executable instructions or computer program are executed by a processor, the method according to any one of claims 1 to 11 is implemented.

Images