Sound detection method, apparatus and computer readable storage medium

Abstract

The application discloses a sound detection method, device and computer readable storage medium. The method comprises the following steps: performing segmented detection on a to-be-detected sound signal to obtain a detection result of each sub-sound signal, wherein the detection result of each sub-sound signal comprises a probability of occurrence of each preset sound event in the sub-sound signal; and determining a sound event occurring in the to-be-detected sound signal based on the detection result of each sub-sound signal. The method provided by the application can improve the accuracy of sound detection.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of sound detection, in particular to a sound detection method, device and computer readable storage medium. BACKGROUND

[0002] In today's world where monitoring is ubiquitous and image detection is increasingly accurate, machines have the ability to detect events occurring in images through image information from monitoring cameras. However, in some corners where monitoring cannot reach, events that endanger people's health and damage people's property still occur, thus giving rise to sound detection. At the present stage, sound detection has been greatly developed, but the accuracy needs to be further improved. SUMMARY

[0003] The present application provides a sound detection method, device and computer readable storage medium, which can improve the accuracy of sound detection.

[0004] The first aspect of the embodiment of the present application provides a sound detection method, which comprises: segmenting and detecting a to-be-detected sound signal to obtain a detection result of each segment of a sub-sound signal, wherein the detection result of each segment of the sub-sound signal comprises a probability of occurrence of each preset sound event in the sub-sound signal; and determining a sound event occurring in the to-be-detected sound signal based on the detection result of each segment of the sub-sound signal.

[0005] The second aspect of the embodiment of the present application provides a sound detection device, which comprises a processor, a memory and a communication circuit, the processor is coupled to the memory and the communication circuit respectively, the memory stores program data, and the processor executes the program data in the memory to realize the steps in the above method.

[0006] The third aspect of the embodiment of the present application provides a computer readable storage medium, which stores a computer program, and the computer program can be executed by a processor to realize the steps in the above method.

[0007] The present application first segments and detects a to-be-detected sound signal to obtain a detection result of each segment of a sub-sound signal, wherein the detection result of each segment of the sub-sound signal comprises a probability of occurrence of each preset sound event in the sub-sound signal, then comprehensively determines a sound event occurring in the to-be-detected sound signal based on the detection result of each segment of the sub-sound signal, and in this process, since multiple detection results are comprehensively determined to obtain a final result, errors in single detection can be avoided, and the accuracy of detection can be improved. BRIEF DESCRIPTION OF DRAWINGS

[0008] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can be obtained without creative labor based on these drawings. Among them:

[0009] Figure 1 is a flowchart of an embodiment of the sound detection method of the present application;

[0010] Figure 2 is a flowchart of step S110 in Figure 1

[0011] Figure 3 is a flowchart of step S111 in Figure 2

[0012] Figure 4 is a partial flowchart of an embodiment of the sound detection method of the present application;

[0013] Figure 5 is a flowchart of step S220 in Figure 4

[0014] Figure 6 is a flowchart of step S120 in an application scenario in Figure 1

[0015] is a structural diagram of an embodiment of the sound detection device of the present application; Figure 7

[0016] Figure 8 is a structural diagram of another embodiment of the sound detection device of the present application;

[0017] Figure 9 is a structural diagram of an embodiment of the computer readable storage medium of the present application. DETAILED DESCRIPTION

[0018] The technical solutions in the embodiments of the present application will be described clearly and completely in the following description of the embodiments of the present application in combination with the drawings. Obviously, the described embodiments are only some embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0019] First, the key terms in the present application are explained:

[0020] ​​​​In the present application, a sound event refers to an event representing a certain behavior with a sound, such as the sound of a certain animal, the sound of an explosion, the sound of a certain device movement, the sound of a vehicle starting, the sound of a horn, etc.; a preset sound event refers to a sound event selected from a plurality of sound events, and the purpose of the present application is to identify whether a preset sound event occurs in a to-be-detected sound signal. The number of preset sound events can be one or more than two.

[0021] Referring to Figure 1 , Figure 1 is a flowchart of an embodiment of the sound detection method of the present application, which comprises:

[0022] S110: segmentally detecting the to-be-detected sound signal to obtain a detection result of each segment of the sub-sound signal, wherein the detection result of each segment of the sub-sound signal comprises the probability of occurrence of each preset sound event in the sub-sound signal.

[0023] In an application scenario, each segment of the sub-sound signal is detected in sequence during the reception of the to-be-detected sound signal to obtain the corresponding detection result; in another application scenario, after the to-be-detected sound signal is received, the to-be-detected sound signal is segmented, and then each segment of the sub-sound signal is detected to obtain the detection result of each segment of the sub-sound signal.

[0024] The detection result of each segment of the sub-sound signal comprises the probability of occurrence of each preset sound event, that is, how many preset sound events, how many probabilities in the detection result.

[0025] For example, assuming that there are three preset sound events, namely cat meowing, dog barking and explosion, the detection result of each segment of the sub-sound signal comprises three probabilities, namely the probability of occurrence of cat meowing, the probability of occurrence of dog barking and the probability of occurrence of explosion in the corresponding sub-sound signal.

[0026] In combination with Figure 2 In the present embodiment, each segment of the sub-sound signal is processed as follows to obtain the detection result of each segment of the sub-sound signal, and the processing process comprises:

[0027] S111: extracting the sound feature of the sub-sound signal.

[0028] In the feature extraction of each segment of the sub-sound signal, the log-mel energy feature of the sub-sound signal can be extracted, or the mel-frequency cepstral coefficient of the sub-sound signal can be extracted.

[0029] Referring to Figure 3 In the present embodiment, the step S111 of extracting the sound feature of the sub-sound signal comprises:

[0030] S1111: extracting the sound feature of each frame of sound signal in the sub-sound signal.

[0031] Frame each sub-sound signal respectively, and then extract the sound feature of each frame sound signal in each sub-sound signal.

[0032] The sound feature of each frame sound signal is a vector, and the dimension of the vector can be determined according to specific conditions. For example, if the discrimination of the sound signal to be detected is low, 64-dimensional log-mel energy features of each frame sound signal can be extracted, and if the discrimination of the sound signal to be detected is high, 13-dimensional mel-frequency cepstral coefficients of each frame sound signal can be extracted.

[0033] S1112: splice the sound feature of each frame sound signal to obtain the sound feature of the sub-sound signal.

[0034] For any sub-sound signal, the sound feature of each frame sound signal corresponding to the sub-sound signal is spliced to obtain a two-dimensional matrix, and the two-dimensional matrix is the sound feature of the sub-sound signal.

[0035] In one application scenario, the number of rows of the two-dimensional matrix is the number of frames of the sound signal included in the corresponding sub-sound signal, and the number of columns is the dimension of the sound feature of each frame sound signal, that is, in the two-dimensional matrix, each row is the sound feature of a frame sound signal, and different rows correspond to different frames of sound signals; in another application scenario, the number of columns of the two-dimensional matrix is the number of frames of the sound signal included in the corresponding sub-sound signal, and the number of rows is the dimension of the sound feature of each frame sound signal, that is, in the two-dimensional matrix, each column is the sound feature of a frame sound signal, and different columns correspond to different frames of sound signals.

[0036] S112: send the sound feature of the sub-sound signal into the pre-trained detection network to obtain the detection result of the sub-sound signal.

[0037] After obtaining the sound feature of each sub-sound signal, in order to improve the detection efficiency and accuracy, the pre-trained detection network is used to identify the sound feature of each sub-sound signal to obtain the detection result of each sub-sound signal.

[0038] The detection network is pre-trained, and after the sound feature of the sub-sound signal is sent into the detection network, the detection network outputs the corresponding detection result.

[0039] In order to reduce the size of the detection network and achieve the purpose of reducing the inference time of the detection network, the detection network is set to include a depth separable convolution layer, a recurrent layer, a pooling layer and a linear layer, and the number of the depth separable convolution layer, the recurrent layer, the pooling layer and the linear layer is multiple, and the pooling layer is connected after each convolution layer.

[0040] ReferenceFigure 4 The training process of the detection network comprises:

[0041] S210: Obtain a training data set, the training data set comprising a plurality of sound signals in which each preset sound event occurs.

[0042] The sound signal of the preset sound event refers to the sound signal in which the preset sound event occurs.

[0043] The training data set comprises sound signals of at least two preset sound events, so that the detection result of each sub-sound signal output by the detection network comprises the probability of occurrence of each of the at least two preset sound events in the sub-sound signal, that is, the detection result of each sub-sound signal comprises at least two probabilities. For ease of description, the number of preset sound events in the training data set is denoted as N, where N is an integer greater than 1.

[0044] Of course, in other embodiments, the training data can only comprise sound signals of one preset sound event, and the detection result of each sub-sound signal comprises only one probability.

[0045] In order to ensure the accuracy of the detection network, the sound signal of each preset sound event is multiple.

[0046] In addition, considering that if the number of sound signals of the preset sound event is unbalanced, the network may be over-fitted, for example, assuming that the training data comprises 10 sound signals of cat meowing, 10 sound signals of dog barking and 1000 sound signals of explosion, no matter how the detection network is trained, when the trained detection network is used for detection, the probability of occurrence of explosion in most of the detection results is relatively large, therefore, in order to avoid the above defects, the training data set is set to comprise at least N training data subsets; the sound signals in one training data subset occur in one of the N preset sound events, and the sound signals in different training data subsets occur in different preset sound events; the ratio of the maximum value to the minimum value of the number of sound signals in the N training data subsets is less than or equal to a first threshold value.

[0047] Specifically, assuming that the number of sound signals of the preset sound event comprising the most sound signals is L1, and the number of sound signals of the preset sound event comprising the least sound signals is L2, the ratio of L1 to L2 is less than or equal to the first threshold value. The first threshold value can be set according to actual conditions, for example, 10 or 5, etc.

[0048] In the embodiment, if there is data imbalance in the obtained training data set, the training data set needs to be expanded before the detection network is trained using the training data set, and the expansion process includes:

[0049] (a) determining the maximum value of the number of sound signals in the N training data subsets.

[0050] (b) in response to the target training data subset in which the number of sound signals is less than the second threshold value, performing data expansion processing on the sound signals in the target training data subset until the ratio of the number of sound signals in the target training data subset to the maximum value is equal to or greater than the second threshold value, and the product of the second threshold value and the first threshold value is 1.

[0051] Wherein, the ratio of the number of sound signals in the target training data subset to the maximum value is less than the second threshold value, which means that the number of sound signals in the target training data subset is small, and the sound signals in the target training data subset need to be expanded until the ratio of the number of sound signals in the target training data subset to the maximum value is equal to or greater than the second threshold value.

[0052] The following examples are used to illustrate the application:

[0053] Suppose the second threshold value is 0.9, and the training data set includes sound signals of preset sound event A, sound signals of preset sound event B, sound signals of preset sound event C, and sound signals of preset sound event D, the number of sound signals of preset sound event A is 100, the number of sound signals of preset sound event B is 90, the number of sound signals of preset sound event C is 60, and the number of sound signals of preset sound event D is 50. That is, the number of training data subsets at this time is 4.

[0054] Then first determine the maximum value of the number of sound signals in the four training data subsets is 100, and since the second threshold value is 0.9, the subsets of preset sound event C and preset sound event D are determined as the target training data subset, and the sound signals of preset sound event C and preset sound event D are increased until the number of sound signals of preset sound event C and preset sound event D is 90.

[0055] In an application scenario, the step (b) of performing data expansion processing on the sound signals in the target training data subset includes: performing frequency shift processing on the sound signals in the target training data subset to obtain new sound signals, and adding the new sound signals in the target training data subset.

[0056] Wherein, for a certain sound signal of a preset sound event, after frequency shifting, the audio of the sound signal can be changed to obtain a new sound signal, which is still the sound signal of the preset sound event.

[0057] Wherein, the process of frequency shifting includes:

[0058] (a1) Fourier transform (for example, short-time Fourier transform) the original sound signal to obtain the frequency spectrum of the original sound signal.

[0059] (b1) Modify the frequency spectrum of the original sound signal by setting parameters.

[0060] (c1) Convert the modified frequency spectrum from the frequency domain to the time domain by inverse Fourier transform to obtain a time-domain waveform, which differs from the original sound signal in length and sampling frequency.

[0061] (d1) Resample the time-domain waveform to make the sampling rate and length of the obtained sound signal consistent with those of the original sound signal, thus obtaining a new sound signal after frequency shifting, which is a subjective change in tone from the original sound signal.

[0062] Wherein, step (b1) can randomly select different parameters to modify the frequency spectrum of the same original sound signal within a certain range, thereby performing different frequency shifts on the same original sound signal to obtain multiple different new sound signals.

[0063] The above frequency shifting process can also be used to perform frequency shifting on multiple sound signals in the above target training data subset to obtain multiple new sound signals.

[0064] In an application scenario, step (b) of data expansion processing of sound signals in the target training data subset includes: using a generative adversarial network to generate similar sound signals of the sound signals in the target training data subset, and adding the similar sound signals to the target training data subset.

[0065] Wherein, the adversarial network includes a generator and a discriminator, the generator generates multiple similar sound signals of the original sound signal, the discriminator determines the difference value between the multiple similar sound signals and the original sound signal, and finally filters out similar sound signals with a difference value below a difference threshold, and adds the filtered similar sound signals to the training data.

[0066] In other application scenarios, other methods can also be used to increase the sound signals of the preset sound event, such as spectral blurring processing of the original sound signal.

[0067] It should be noted that ensuring data balance of the training data set is not a necessary step. In other embodiments, any method in the prior art can be used to generate the training data set when generating the training data set.

[0068] S220: training the detection network using the training data set to obtain a trained detection network.

[0069] In this embodiment, in order to make the trained detection model closer to the actual application scenario, referring to Figure 5 , step S220 includes:

[0070] S221: based on the reference audio associated with the target scene, performing interference processing on the sound signal in the training data set, so that the distribution characteristics of the sound signal are consistent with the target scene.

[0071] The target scene refers to the scene in which the sound signal to be detected is located, that is, the purpose of the present application is to detect the sound event occurring in the sound signal to be detected collected from the target scene.

[0072] It can be understood that when the target scene of the sound signal to be detected is different, different reference audios need to be used when training the detection model.

[0073] Among them, the reference audio associated with the target scene ensures the characteristics unique to the target scene. For example, when the target scene is a station, the reference audio can be a sound signal of a whistle.

[0074] After using the reference audio associated with the target scene to perform interference processing on the sound signal, the sound signal carries the characteristics representing the target scene, thereby improving the recognition accuracy of the detection network for the sound signal to be detected in the target scene.

[0075] In an application scenario, the reference audio includes an impulse response associated with the target scene, and at this time step S221 performs convolution processing on the sound signal using the impulse response associated with the target scene.

[0076] Among them, all or part of the sound signals in the training data set are respectively convolved with the impulse response associated with the target scene to increase the robustness of the detection network and improve the detection accuracy.

[0077] In other application scenarios, when the target scene of the sound signal to be detected cannot be predicted, in order to enable the detection network to adapt to various different complex environments, the sound signal can also be convolved with the impulse responses of different environments at this time.

[0078] In another application scenario, the reference audio includes noise associated with the target scene, and at this time step S221 performs superposition processing on the sound signal using the noise.

[0079] Wherein, assuming that the original sound signal is s, the superimposed noise signal is n, the new signal after superposition is x, and the signal-to-noise ratio is SNR, the process of superimposing the noise signal n on the original sound signal s includes:

[0080] First, the energy E of the original signal is calculated s And the energy E of the noise signal n :

[0081] E s =s 2 ;E n =n 2 ;

[0082] Then, the noise signal adjustment multiple A is calculated according to the signal-to-noise ratio, because Therefore

[0083] Finally, the noise signal is superimposed on the original sound signal according to the adjustment multiple A to obtain the new signal x:

[0084]

[0085] Among them, one or more noises of the target scene are obtained, and then different sound signals can be superimposed with the same noise at different signal-to-noise ratios, or different sounds can also be superimposed with different noises.

[0086] In other application scenarios, when the scene of the sound signal to be detected is unknown, in order to enable the detection network to adapt to various different complex environments, in step S221, the sound signal is superimposed with noises of different environments.

[0087] Among them, only the impulse response associated with the target scene and the sound signal can be used for convolution processing, or only the noise and the sound signal can be used for superposition processing, or both the impulse response associated with the target scene and the sound signal can be used for convolution processing, and the noise and the sound signal can be used for superposition processing, and the convolution can be performed first, then the superposition, or the superposition can be performed first, then the convolution.

[0088] S222: training the detection network using the training data set after interference processing.

[0089] The detection network is detected using the training data set after interference processing until the detection network converges.

[0090] Through the above method, the detection result of each sub-sound signal is obtained, and then based on the detection result of each sub-sound signal, the sound event occurring in the sound signal to be detected is determined, which can be specifically referred to as follows.

[0091] S120: Determine the sound event occurred in the to-be-detected sound signal based on the detection result of each sub-sound signal.

[0092] In an application scenario, step S120 specifically includes: determining the target sound event as the sound event occurred in the to-be-detected sound signal, wherein the target sound event is the sound event in the preset sound events, and the number of times that the probability corresponding to the target sound event exceeds the third threshold value in all detection results of the sub-sound signals exceeds the number threshold value.

[0093] Suppose the to-be-detected sound signal includes M sub-sound signals, since each sub-sound signal corresponds to a detection result, there are M detection results, at this time, if the number of times that the probability corresponding to a preset sound event exceeds the third threshold value in the M detection results is greater than or equal to the number threshold value, it is determined that the preset sound event occurred in the to-be-detected sound signal. The third threshold value and the number threshold value can be set according to actual conditions, which are not limited here.

[0094] For example, M = 2a + 1, where a is an integer greater than zero, that is, the to-be-detected sound signal includes an odd number of sub-sound signals, at this time, the number threshold value in step S120 can be set to a + 1, that is, if the number of times that the probability of a preset sound event exceeds the third threshold value in 2a + 1 detection results is greater than or equal to a + 1, it is determined that the preset sound event occurred in the to-be-detected sound signal.

[0095] In this application scenario, considering that multiple sub-sound signals belong to the to-be-detected sound signal, the probability corresponding to the same preset sound event in the detection results of adjacent two sub-sound signals may be different, but will not differ much. In order to avoid the probability corresponding to the same preset sound event in the detection results of adjacent two sub-sound signals suddenly changing due to errors in the detection process, step S120 will also perform smoothing processing on the detection results of all sub-sound signals before determining that the target sound event occurred in the to-be-detected sound signal, to ensure that the difference between the probabilities corresponding to the same preset sound event in the detection results of adjacent two sub-sound signals is within a preset range.

[0096] The smoothing processing can be median filtering smoothing, Gaussian filtering smoothing, etc., which is not limited here.

[0097] In another application scenario, referring to Figure 6 , step S120 includes:

[0098] S121: Send the detection results of all sub-sound signals to the integration network to obtain an integration result, and the integration result includes the confidence of each preset sound event occurred in the to-be-detected sound signal.

[0099] S122: Determine the sound event occurred in the to-be-detected sound signal according to the integration result.

[0100] Still taking that the to-be-detected sound signal includes M sub-sound signals as an example:

[0101] In step S121, the detection results of the M sub-sound signals are all sent into the integration network to obtain an integration result, which includes the confidence degrees of the occurrence of each preset sound event in the to-be-detected sound signal. It can be understood that the number of confidence degrees in the integration result is equal to the number of confidence degrees in the detection results of the sub-sound signals.

[0102] The integration network can be an LSTM or GRU network, which is not limited here.

[0103] The integration network receives the detection results of all sub-sound signals, splices all the detection results into a matrix, and then integrates the matrix into a probability group, i.e., obtains the integration result.

[0104] The process of integrating the matrix into the probability group can be: calculating the average, maximum or minimum value of the probability corresponding to each preset sound event, and then integrating the average, maximum or minimum value corresponding to each preset sound event into a probability group, i.e., the integration result.

[0105] Since the integration network has a strong time sequence modeling capability, it can accurately and efficiently convert the detection results of all sub-sound signals into the integration result.

[0106] After obtaining the integration result, step S122 determines the sound event occurred in the to-be-detected sound signal according to the integration result. The process can be specifically: determining the maximum value of the confidence degrees in the integration result; and determining the preset sound event corresponding to the maximum value as the sound event occurred in the to-be-detected sound signal.

[0107] Specifically, the maximum confidence degree is found in the integration result, and then the preset sound event corresponding to the maximum confidence degree is determined as the sound event occurred in the to-be-detected sound signal.

[0108] It is considered that when no preset sound event occurs in the to-be-detected sound signal, if the preset sound event corresponding to the maximum value in the integration result is determined as the sound event occurred in the to-be-detected sound signal, it is not in line with the actual situation.

[0109] Therefore, before determining the preset sound event corresponding to the maximum value as the sound event occurred in the sound signal to be detected, it is further needed to determine the size relationship between the maximum value and the fourth threshold value. If the maximum value is greater than or equal to the fourth threshold value, the preset sound event corresponding to the maximum value is determined as the sound event occurred in the sound signal to be detected. If the maximum value is less than the fourth threshold value, it is determined that the preset sound event does not occur in the sound signal to be detected.

[0110] The fourth threshold value can be set according to actual conditions, for example, set to 0.5 or 0.8, etc.

[0111] Herein, the fourth threshold value is assumed to be 0.8. If the integration result is (0.5, 0.6, 0.7), since the maximum value 0.7 in the integration result does not exceed 0.8, it is determined that the preset sound event does not occur in the sound signal to be detected. If the integration result is (0.9, 0.5, 0.6), since the maximum value 0.9 in the integration result exceeds 0.8, it is determined that the preset sound event corresponding to 0.9 occurs in the sound signal to be detected.

[0112] After obtaining the integration result, the sound event occurred in the sound signal to be detected is determined according to the integration result. The process can further include determining the confidence value greater than the fifth threshold value in the integration result, and determining the preset sound event corresponding to the determined confidence value as the sound event occurred in the sound signal to be detected.

[0113] The fifth threshold value can be set according to actual needs, for example, set to 0.8. It can be understood that if there is no confidence value greater than the fifth threshold value in the integration result, it is determined that the preset sound event does not occur in the sound signal to be detected.

[0114] Referring to Figure 7 , Figure 7 is a structural schematic diagram of an embodiment of the sound detection device. The sound detection device 200 includes a processor 210, a memory 220, and a communication circuit 230. The processor 210 is coupled to the memory 220 and the communication circuit 230, respectively. The memory 220 stores program data. The processor 210 implements the steps in the method of any of the above embodiments by executing the program data in the memory 220. The detailed steps can be referred to in the above embodiments, which will not be described herein.

[0115] The sound detection device 200 can be a computer, a mobile phone, or any device with algorithm processing capability, which is not limited herein.

[0116] Referring to Figure 8 , Figure 8is a structural schematic diagram of another embodiment of the sound detection device. The sound detection device 300 includes a detection module 310 and a determination module 320.

[0117] The detection module 310 is configured to perform segmented detection on the to-be-detected sound signal to obtain a detection result of each segment of the sub-sound signal, where the detection result of each segment of the sub-sound signal includes a probability of occurrence of each preset sound event in the sub-sound signal.

[0118] The determination module 320 is connected with the detection module 310 and is configured to determine a sound event occurring in the to-be-detected sound signal based on the detection result of each segment of the sub-sound signal.

[0119] The sound detection device 300 performs the steps in the sound detection method in any one of the above embodiments when working, and the detailed steps can be referred to the above embodiments and will not be repeated here.

[0120] The sound detection device 300 can be any device with algorithm processing capability such as a mobile phone or a computer, which is not limited here.

[0121] Referring to Figure 9 , Figure 9 is a structural schematic diagram of an embodiment of the computer readable storage medium. The computer readable storage medium 400 stores a computer program 410, and the computer program 410 can be executed by a processor to implement the steps in any one of the above methods.

[0122] The computer readable storage medium 400 can be a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, or any device that can store the computer program 410, or can be a server that stores the computer program 410, which can send the stored computer program 410 to other devices for running, or can run the stored computer program 410 itself.

[0123] The above is only an embodiment of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation, or direct or indirect application in other related technical fields, is also included in the patent protection scope of the present application.

Claims

1. A sound detection method, characterized in that, The method includes: The sound signal to be detected is segmented for detection to obtain the detection result of each sub-sound signal. The detection result of each sub-sound signal includes the probability of each preset sound event occurring in the sub-sound signal. The detection result of the sub-sound signal is obtained by the sound features of the sub-sound signal through a pre-trained detection network. The detection network is trained on a training dataset, which includes N training data subsets. The sound signal in each training data subset occurs one preset sound event. The ratio of the maximum to the minimum number of sound signals in the N training data subsets is less than or equal to a first threshold. Based on the detection results of each sub-sound signal, the sound events occurring in the sound signal to be detected are determined; Before training the detection model using the training dataset, if a target training data subset exists in the N training data subsets, the number of sound signals in the target training data subset is expanded to a ratio equal to or greater than the maximum value, wherein the ratio of the number of sound signals in the target training data subset to the maximum value is less than the second threshold, and the product of the second threshold and the first threshold is 1.

2. The method according to claim 1, characterized in that, The step of segmenting the sound signal to be detected and obtaining the detection result of each sub-segment of the sound signal includes: The sub-sound signals of each segment are processed as follows: Extract the acoustic features of the sub-sound signal; The acoustic features of the sub-sound signal are fed into a pre-trained detection network to obtain the detection result of the sub-sound signal.

3. The method according to claim 2, characterized in that, The step of extracting the acoustic features of the sub-sound signal includes: Extract the acoustic features of each frame of the sub-sound signal; The sound features of each frame of sound signal are spliced ​​together to obtain the sound features of the sub-sound signal.

4. The method according to claim 2, characterized in that, Before segmenting the sound signal to be detected and obtaining the detection result of each sub-segment, the method further includes: Obtain a training dataset, which includes multiple sound signals that occur for each of the preset sound events; The detection network is trained using the training dataset to obtain the trained detection network.

5. The method according to claim 4, characterized in that, The step of training the detection network using the training dataset to obtain the trained detection network includes: Based on the reference audio associated with the target scene, the sound signals in the training dataset are subjected to interference processing so that the distribution characteristics of the sound signals match the target scene; The detection network is trained using the training dataset after the interference processing.

6. The method according to claim 5, characterized in that, The step of interfering with the sound signals in the training dataset based on reference audio associated with the target scene includes: The reference audio includes the impulse response associated with the target scene, and convolution processing is performed using the impulse response associated with the target scene and the audio signal; and / or The reference audio includes noise associated with the target scene, and the noise is superimposed on the sound signal.

7. The method according to claim 4, characterized in that, The preset sound events include N; where N is an integer greater than 1. The training dataset includes at least N subsets of training data; The sound signal in one subset of the training data occurs one of N preset sound events, and the preset sound events that occur are different for the sound signals in different subsets of the training data. The ratio of the maximum to the minimum number of sound signals in N training data subsets is less than or equal to the first threshold.

8. The method according to claim 1, characterized in that, The step of performing data expansion processing on the sound signals in the target training data subset includes: The sound signals in the target training data subset are subjected to frequency shifting to obtain new sound signals, and the new sound signals are added to the target training data subset. Alternatively, a generative adversarial network can be used to generate similar sound signals to the sound signals in the target training data subset, and the similar sound signals can be added to the target training data subset.

9. The method according to claim 2, characterized in that, The detection network includes depthwise separable convolutional layers, recurrent layers, pooling layers, and linear layers, wherein there are multiple depthwise separable convolutional layers, recurrent layers, pooling layers, and linear layers, and each depthwise separable convolutional layer is connected to a pooling layer.

10. The method according to any one of claims 1-9, characterized in that, The step of determining the sound event occurring in the sound signal to be detected based on the detection results of each segment of the sub-sound signal includes: The target sound event is determined as the sound event occurring in the sound signal to be detected, wherein the target sound event is the sound event among the various preset sound events, and in the detection results of all the sub-sound signals, the number of times the probability corresponding to the target sound event exceeds the third threshold exceeds the number threshold.

11. The method according to claim 10, characterized in that, Before determining the target sound event as a sound event occurring in the sound signal to be detected, the method further includes: The detection results of all the sub-sound signals are smoothed so that the difference in the probability of the same preset sound event in the detection results of two adjacent sub-sound signals is within a preset range.

12. The method according to any one of claims 1-9, characterized in that, The step of determining the sound event occurring in the sound signal to be detected based on the detection results of each segment of the sub-sound signal includes: The detection results of all the sub-sound signals are sent into the integration network to obtain the integration result, which includes the confidence level of each preset sound event occurring in the sound signal to be detected. Based on the integration results, the sound events occurring in the sound signal to be detected are determined.

13. The method according to claim 12, characterized in that, The step of determining the sound events occurring in the sound signal to be detected based on the integration result includes: Determine the maximum confidence level in the integrated results; The preset sound event corresponding to the maximum value is determined as the sound event occurring in the sound signal to be detected.

14. The method according to claim 13, characterized in that, Before determining the preset sound event corresponding to the maximum value as the sound event occurring in the sound signal to be detected, the method further includes: Determine the relationship between the maximum value and the fourth threshold; The step of determining the preset sound event corresponding to the maximum value as the sound event occurring in the sound signal to be detected includes: In response to the maximum value being greater than or equal to the fourth threshold, the preset sound event corresponding to the maximum value is determined as a sound event occurring in the sound signal to be detected; or, in response to the maximum value being less than the fourth threshold, it is determined that the preset sound event has not occurred in the sound signal to be detected.

15. The method according to claim 12, characterized in that, The step of determining the sound events occurring in the sound signal to be detected based on the integration result includes: Determine the confidence level of the integrated results that is greater than the fifth threshold; The preset sound event corresponding to the determined confidence level is identified as the sound event occurring in the sound signal to be detected.

16. A sound detection device, characterized in that, The sound detection device includes a processor, a memory, and a communication circuit. The processor is coupled to the memory and the communication circuit. The memory stores program data. The processor executes the program data in the memory to implement the steps of the method as described in any one of claims 1-15.

17. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that can be executed by a processor to implement the steps of the method as described in any one of claims 1-15.

Images