Environment easy-to-mix sound recognition method and system based on double-layer model, and storage medium

Abstract

The invention discloses an environment easy-to-mix sound identification method and system based on a double-layer model and a storage medium. The method comprises the following steps: converting an audio time sequence into a spectrogram through Fourier transform with a time window; image features in the spectrogram are extracted through an artificial intelligence model, the type of a sound source is recognized, if the type of the sound source is a sound source not prone to confusion, a sound source recognition result is directly output, and if the type of the sound source is a sound source prone to confusion, the corresponding spectrogram is input into a recognition model of the sound source prone to confusion to be recognized, the easy-to-confuse sound source identification model is different from the artificial intelligence model in that only the easy-to-confuse sound source is used for training and identification. According to the method, the accuracy of current environment sound source recognition is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of sound source recognition, and in particular to a method, system and storage medium for identifying easily mixed sounds in an environment based on a double-layer model. Background Art

[0002] As my country's urbanization rate continues to increase, urban noise has gradually become a major concern for urban residents. Automatic noise monitoring is an objective means of reflecting the quality level of the urban sound environment. The current evaluation method using sound pressure level as a single indicator leads to misjudgment of the quality of the urban sound environment by natural sounds. It is necessary to classify and identify different environmental sound sources in noise signals. In addition, in recent years, noise complaints have been at the top of the list of environmental complaints in major cities. In order to effectively handle noise complaints and scientifically formulate noise pollution control plans, the first task is to trace and identify the noise sources. In view of the complex and time-sensitive characteristics of urban noise sources, an environmental sound source signal extraction and identification method based on artificial intelligence models is developed. However, there are easily confused sound sources in the environmental sound sources, such as wind and aircraft noise. Existing sound source identification models are currently unable to effectively solve this problem. Summary of the Invention

[0003] To address the above problems, the present invention provides a method, system and storage medium for identifying environmental confusable sounds based on a double-layer model, aiming to use the double-layer model to identify the categories of confusable environmental sound sources and improve the accuracy of current environmental sound source identification.

[0004] According to a first aspect of an embodiment of the present disclosure, a method for identifying ambient confusable sounds based on a two-layer model is provided, the method comprising the following steps:

[0005] Convert the audio time series into a spectrogram using Fourier transform with a time window;

[0006] The artificial intelligence model is used to extract image features from the spectrogram and identify the sound source category. If the sound source category is an unconfusable sound source, the sound source identification result is directly output. If the sound source category is a confusable sound source, the corresponding spectrogram is input into the confusable sound source identification model for identification. The difference between the confusable sound source identification model and the artificial intelligence model is that only confusable sound sources are used for training and identification.

[0007] In some embodiments, converting the audio time series into a spectrogram by Fourier transform with a time window includes the following steps:

[0008] Convert the mechanical vibration signal into an analog current signal through a microphone;

[0009] Convert the continuous analog signal into a digital discrete time series signal through the audio collector;

[0010] Slicing the discrete time series signal into audio time series vectors through a sliding time window, wherein the audio time series vectors have overlapping parts in the time domain;

[0011] Perform Fourier transform on each audio time series vector and remove the phase angle result. Stack the audio time series results after Fourier transform in the direction of time order to obtain a spectrogram.

[0012] In some embodiments, the artificial intelligence model is used to implement the following steps:

[0013] Slice the spectrum graph to obtain n subgraphs with overlapping parts;

[0014] Reshape the subgraph into a one-dimensional array through linear transformation and add the sound source category feature vector with trainable parameters;

[0015] The sub-graph position information vector is further merged into a one-dimensional array to obtain a feature matrix;

[0016] The feature matrix is ​​input into the self-attention encoder to extract the pattern information in the spectrogram.

[0017] In some embodiments, the feature matrix is ​​input to a self-attention encoder to extract pattern information from the spectrogram, wherein the self-attention encoder includes L sub-encoder modules, each of which sequentially implements the following steps:

[0018] Input the feature matrix into the first residual neural network and output the second feature matrix;

[0019] The second feature matrix is ​​input into the second residual neural network, and the third feature matrix is ​​output. The third feature matrix is ​​input into the next sub-encoder module as the feature matrix;

[0020] Among them, the first residual neural network and the second residual neural network each include a regularization neural network layer and a multi-head attention mechanism neural network layer.

[0021] In some embodiments, the multi-head attention mechanism neural network layer is implemented including the following steps:

[0022] Execute the following steps h times in parallel:

[0023] Use three linear transformation matrices to multiply the input feature matrix respectively to obtain the query Q, key value K and value V;

[0024] Multiply the query Q by the key value K, divide it by the square root of the number of columns in the matrix, normalize it using the exponential function, and multiply the output by the value V;

[0025] The results of h parallel executions are horizontally concatenated to obtain a new matrix, and the new matrix is ​​linearly transformed using a linear transformation matrix.

[0026] In some embodiments, the specific expression of the regularized neural network layer is: Where i = 1, 2, ..., N + 1, N represents the length of the sliding time window, Represents the sound source category feature vector of the i-th subgraph in the l-th sub-encoder module, E[] represents the calculation of mathematical expectation, Var[] represents the calculation of variance, ∈ is a very small amount to prevent the calculation of variance from being zero, resulting in the denominator being zero, γ and β represent the trainable parameters of the regular layer neural network.

[0027] In some embodiments, the size of the sub-image is set to 3×32×32.

[0028] According to a second aspect of an embodiment of the present disclosure, a system for identifying ambient confusable sounds based on a two-layer model is provided, the system comprising:

[0029] Spectrum acquisition module, used to convert audio time series into spectrogram through Fourier transform with time window;

[0030] The two-layer model recognition module extracts image features from the spectrogram through an artificial intelligence model and identifies the sound source category. If the sound source category is an unconfusable sound source, the sound source recognition result is directly output. If the sound source category is a confusable sound source, the corresponding spectrogram is input into the confusable sound source recognition model for recognition. The difference between the confusable sound source recognition model and the artificial intelligence model is that only confusable sound sources are used for training and recognition.

[0031] According to a third aspect of an embodiment of the present disclosure, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of the above-mentioned method for identifying environmental easily mixed sounds based on a two-layer model are implemented.

[0032] According to a fourth aspect of an embodiment of the present disclosure, a non-transitory computer-readable storage medium is provided, on which computer instructions are stored. When the instructions are executed by a processor, the steps of the above-mentioned method for identifying environmental easily mixed sounds based on a two-layer model are implemented.

[0033] The embodiments of the present disclosure provide a method, system, and storage medium for identifying easily confused ambient sounds based on a double-layer model. The double-layer model is used to perform secondary classification of easily confused sound sources in ambient sounds, and a self-attention mechanism is used to identify the categories of ambient sound sources, thereby improving the current accuracy of ambient noise source monitoring.

[0034] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0036] Figure 1 This is a flow chart of a method for identifying easily mixed sounds in an environment based on a double-layer model in an embodiment of the present invention;

[0037] Figure 2 2 is a schematic diagram showing the principle of a method for identifying easily mixed sounds in an environment based on a double-layer model in an embodiment of the present invention;

[0038] Figure 3 Schematic diagram of the process of converting time series audio into a spectrogram in an embodiment of the present invention;

[0039] Figure 4 1 is a schematic diagram of a process for identifying a sound source category by using image features in a spectrogram according to an embodiment of the present invention;

[0040] Figure 5 Schematic diagram of the implementation principle of the self-attention encoder in an embodiment of the present invention;

[0041] Figure 6 Schematic diagram of the implementation principle of the multi-head attention neural network in an embodiment of the present invention;

[0042] Figure 7 is a schematic diagram of a method for mapping spectrogram pattern features to environmental sound source categories in an embodiment of the present invention;

[0043] Figure 8 1 is a schematic diagram of the structure of an environmental easily mixed sound recognition system based on a double-layer model in an embodiment of the present invention;

[0044] Figure 9 It is a schematic diagram of an electronic device in an embodiment of the present invention. DETAILED DESCRIPTION

[0045] The present invention will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are intended only to illustrate the present invention and are not intended to limit the present invention. It should also be noted that, for ease of description, the accompanying drawings only illustrate portions relevant to the present invention, not all structures.

[0046] Before discussing the exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flow charts. Although the flow charts describe the steps as sequential processes, many of the steps can be implemented in parallel, concurrently, or simultaneously. In addition, the order of the steps can be rearranged. The process can be terminated when its operation is completed, but can also have additional steps not included in the accompanying drawings. The process can correspond to a method, function, procedure, subroutine, subprogram, etc.

[0047] The present invention provides a method, system, and storage medium for identifying easily mixed sounds in an environment based on a double-layer model, including the following embodiments:

[0048] The environmental mixed sound recognition method based on the double-layer model, such as Figure 1 、 Figure 2 As shown, the method includes the following steps:

[0049] S1, convert the audio time series into a spectrogram through Fourier transform with time window;

[0050] S2. Extract image features from the spectrogram through the artificial intelligence model and identify the sound source category. If the sound source category is an unconfusable sound source, directly output the sound source identification result. If the sound source category is a confusable sound source, input the corresponding spectrogram into the confusable sound source identification model for identification. The difference between the confusable sound source identification model and the artificial intelligence model is that only confusable sound sources are used for training and identification.

[0051] In the specific implementation process, the microphone and audio collector are used to convert the sound and mechanical vibration signals in the ambient sound into discrete time series electrical signals.

[0052] In S1, the audio time series is converted into a spectrogram by Fourier transform with a time window, such as Figure 3 As shown, the following steps are included:

[0053] Convert the mechanical vibration signal into an analog current signal through a microphone;

[0054] Convert the continuous analog signal into a digital discrete time series signal through the audio collector;

[0055] Slicing the discrete time series signal into audio time series vectors through a sliding time window, wherein the audio time series vectors have overlapping parts in the time domain;

[0056] Perform Fourier transform on each audio time series vector and remove the phase angle result. Stack the audio time series results after Fourier transform in the direction of time order to obtain a spectrogram.

[0057] In the specific implementation process, for the digitized discrete ambient sound signal s d (i) Using a sliding time window, a signal is cut into different time slices. The processing process is:

[0058] s(j,k)=s d (j+k(N-1))

[0059] Where k is the index of the time slice, j is the index of each frame in the slice, and N is the length of the sliding time window. The audio time series in each time window is processed by a window function to balance the frequency leakage caused by the audio time series in the time window not meeting the signal stationarity assumption.

[0060] Perform discrete Fourier transform on the windowed signal

[0061]

[0062] where k = 0, 1, 2, ..., N f -1 represents discrete frequency, N f is the number of discrete signal frames within the time slice.

[0063] Based on S(n,k), the signal is converted into a spectrum diagram, and the result is The spectrum graph is divided into n subgraphs with overlapping parts.

[0064] The artificial intelligence model is used to implement the following steps:

[0065] The spectrum graph is sliced ​​to obtain n subgraphs with overlapping parts. The size of the subgraphs is preferably 3×32×32.

[0066] Reshape the subgraph into a one-dimensional array through linear transformation and add the sound source category feature vector with trainable parameters;

[0067] The sub-graph position information vector is further merged into a one-dimensional array to obtain a feature matrix;

[0068] The feature matrix is ​​input into the self-attention encoder to extract the pattern information in the spectrogram.

[0069] In the specific implementation process, Figure 4 As shown, the pixel matrix of n sub-images is transformed into a vector through matrix multiplication, and an additional category feature with all trainable parameters is added to obtain the input feature and

[0070] The position information of the sub-image before slicing is integrated into the feature vector to obtain a new input feature. and

[0071] Will The input is fed into the self-attention encoder to extract the spectrogram pattern features.

[0072] The spectrogram pattern features are mapped to the environmental sound source categories through a fully connected neural network layer, such as Figure 7 As shown in the figure, the specific mapping process is that the elements of different dimensions in the spectrum pattern features are used as input neurons. After calculation through the hidden layer, several output neurons are obtained. The values ​​of these output neurons are the confidence levels of each sound source category.

[0073] The feature matrix is ​​input into the self-attention encoder to extract the pattern information in the spectrogram. The self-attention encoder includes L sub-encoder modules, and each sub-encoder module sequentially implements the following steps:

[0074] Input the feature matrix into the first residual neural network and output the second feature matrix;

[0075] The second feature matrix is ​​input into the second residual neural network, and the third feature matrix is ​​output. The third feature matrix is ​​input into the next sub-encoder module as the feature matrix;

[0076] Among them, the first residual neural network and the second residual neural network each include a regularization neural network layer and a multi-head attention mechanism neural network layer.

[0077] In the specific implementation process, Figure 5 As shown in Figure 2, the self-attention encoder consists of L sub-encoder modules. The output of the previous encoder is the input of the next encoder. The specific process is:

[0078] Input features Input to the first sub-encoder module, which consists of two residual neural network modules. The residual part of the first residual neural network module contains a regularization neural network layer and a multi-head attention neural network calculation layer, as shown below:

[0079]

[0080] Where l = 1, 2, ... L, represents the lth sub-encoder module, Represents the output of the previous sub-encoder module or the input feature received by the first autoencoder module, MSA represents the multi-head attention neural network layer, and LN represents the regularized neural network layer. The calculation process is as follows:

[0081]

[0082] Among them, i=1,2,…,N+1 represents the feature vector of the sound source category Sequence number, N represents the length of the sliding time window, represents the sound source category feature vector of the ith subgraph in the lth sub-encoder module, E[] represents the calculation of mathematical expectation, Var[] represents the calculation of variance, ∈ is a very small amount to prevent the calculation of variance from being zero, resulting in the denominator being zero, preferably 10 -5 is a value, γ and β represent the trainable parameters of the regular layer neural network.

[0083] will output As the input vector of the second residual neural network module, the residual part of the residual neural network module consists of a regularized neural network layer and a multi-layer perceptron neural network layer that are the same as the previous sub-encoder module, as shown below:

[0084]

[0085] in, is the output of the previous residual neural network module, and MLP represents the calculation of the multi-layer perceptron neural network layer.

[0086] The implementation of the multi-head attention neural network layer includes the following steps:

[0087] Execute the following steps h times in parallel:

[0088] Use three linear transformation matrices to multiply the input feature matrix respectively to obtain the query Q, key value K and value V;

[0089] Multiply the query Q by the key value K, divide it by the square root of the number of columns in the matrix, normalize it using the exponential function, and multiply the output by the value V;

[0090] The results of h parallel executions are horizontally concatenated to obtain a new matrix, and the new matrix is ​​linearly transformed using a linear transformation matrix.

[0091] In the specific implementation process, Figure 6 As shown, the calculation is a parallel calculation of h attention algorithms, preferably h = 12. The specific calculation process of the multi-head attention neural network is:

[0092] The input matrix Through the linear transformation matrix whose elements are all trainable neural network parameters Among them D h Generally, 3072 / h=256 is taken, and the calculation results are expressed as Where i=1,2,…,h is the computation path number of different parallel computation processes.

[0093] based on Calculate the attention matrix As shown below:

[0094]

[0095] Among them, softmax() is the normalization function calculated by the neural network.

[0096] H i The new matrix is ​​concatenated in the row direction and linearly transformed by multiplying the linear change matrix whose elements are all trainable parameters, as shown below:

[0097]

[0098] in, is the calculation result of the multi-head attention neural network layer, is a matrix whose elements are trainable parameters.

[0099] In a specific embodiment, Figure 2 As shown, the following steps are included:

[0100] Convert the mechanical vibration signal generated by sound in the environment into a spectrum diagram;

[0101] The spectrogram is input into the artificial intelligence neural network model to identify the sound source. If it is a pre-defined, non-confusable sound source, including thunder, dog barking, running water, chicken crowing, traffic, insects, birdsong, construction, frog croaking, rain, and music, it will be output as the recognition result;

[0102] If the sound source being identified is easily confused, such as airplane and wind, the spectrogram is fed into the easily confused source recognition model. This recognition model differs from the first-layer model, the artificial intelligence neural network model, in that it is trained only on wind and airplane sound datasets and uses this model to identify airplane and wind sounds.

[0103] Another embodiment is used to illustrate a dual-layer model-based environment easily mixed sound recognition system. Figure 8 As shown, the system 800 includes:

[0104] A spectrogram acquisition module 810 is configured to convert an audio time series into a spectrogram by Fourier transform with a time window;

[0105] The dual-layer model recognition module 820 extracts image features from the spectrogram through an artificial intelligence model and identifies the sound source category. If the sound source category is an unconfusable sound source, the sound source recognition result is directly output. If the sound source category is a confusable sound source, the corresponding spectrogram is input into the confusable sound source recognition model for recognition. The difference between the confusable sound source recognition model and the artificial intelligence model is that only confusable sound sources are used for training and recognition.

[0106] In addition to the above modules, the system 800 may also include other components. However, since these components are irrelevant to the content of the embodiment of the present disclosure, their illustration and description are omitted here.

[0107] The other specific working processes of the dual-layer model-based environment easily mixed sound recognition system 800 refer to the description of the dual-layer model-based environment easily mixed sound recognition method embodiment, which will not be repeated here.

[0108] Another embodiment is used to illustrate that the system of the present invention can also be used with the help of Figure 9 The architecture of the computing device shown is implemented. Figure 9 The architecture of the computing device is shown in FIG. Figure 9 As shown, a computer system 910, a system bus 930, one or more CPUs 940, an input / output 920, a memory 950, etc. The memory 950 can store various data or files used for computer processing and / or communication, as well as program instructions executed by the CPU, including the method for identifying easily mixed sounds in an environment based on a dual-layer model according to an embodiment. Figure 9 The architecture shown is only exemplary and may be adjusted based on actual needs when implementing different devices. Figure 9 One or more components in the system. The memory 950, as a computer-readable storage medium, can be used to store software programs, computer executable programs, and modules, such as the program instructions / modules corresponding to the dual-layer model-based environmental confusable sound recognition method in the embodiment of the present invention (for example, the spectrogram acquisition module 810 and the dual-layer model recognition module 820 in the dual-layer model-based environmental confusable sound recognition system 800). One or more CPUs 940 execute various functional applications and data processing of the system of the present invention by running the software programs, instructions, and modules stored in the memory 950, that is, to implement the aforementioned dual-layer model-based environmental confusable sound recognition method, which includes the following steps:

[0109] Convert the audio time series into a spectrogram using Fourier transform with a time window;

[0110] The artificial intelligence model is used to extract image features from the spectrogram and identify the sound source category. If the sound source category is an unconfusable sound source, the sound source identification result is directly output. If the sound source category is a confusable sound source, the corresponding spectrogram is input into the confusable sound source identification model for identification. The difference between the confusable sound source identification model and the artificial intelligence model is that only confusable sound sources are used for training and identification.

[0111] Of course, the processor of the server provided by the embodiment of the present invention is not limited to executing the above-mentioned method operations, but can also execute relevant operations in the environmental confusable sound recognition method based on the double-layer model provided by any embodiment of the present invention.

[0112] The memory 950 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system and applications required for at least one function; the data storage area may store data created based on the use of the terminal, etc. In addition, the memory 950 may include a high-speed random access memory and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some instances, the memory 950 may further include a memory remotely located relative to one or more CPUs 940, and these remote memories may be connected to the device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0113] The input / output 920 may be used to receive input digital or character information and generate key signal input related to user settings and function control of the device. The input / output 920 may also include a display device such as a display screen.

[0114] The embodiment of the present invention also provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the method for identifying environmental fusible sounds based on a double-layer model described in the above embodiment is implemented. The computer-readable storage medium of the embodiment of the present invention can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device or device.

[0115] A computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0116] The program code contained on the storage medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0117] In addition, other specific working processes of a non-transitory computer-readable storage medium refer to the description of the embodiment of the environmental confusable sound recognition method based on the double-layer model, and are not repeated here.

[0118] According to the above-mentioned embodiments, a method, system and storage medium for identifying easily confused environmental sounds based on a double-layer model are provided. A double-layer model is used to perform secondary classification of easily confused sound sources in environmental sounds, and a self-attention mechanism is used to identify the categories of environmental sound sources, thereby improving the current accuracy of environmental noise source monitoring.

[0119] In this document, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a step or method that comprises a series of elements includes not only those elements, but also includes other elements not expressly listed, or also includes elements inherent to such step or method.

[0120] The above is a further detailed description of the present invention in conjunction with specific preferred embodiments, and the specific implementation of the present invention should not be considered to be limited to these descriptions. For those skilled in the art of the present invention, without departing from the concept of the present invention, several simple deductions or substitutions can be made, which should be considered to fall within the scope of protection of the present invention.

Claims

1. A method for identifying easily mixed sounds in an environment based on a double-layer model, characterized in that: The method comprises the following steps: Convert the audio time series into a spectrogram using Fourier transform with a time window; The artificial intelligence model is used to extract image features from the spectrogram and identify the sound source category. If the sound source category is an unconfusable sound source, the sound source identification result is directly output. If the sound source category is a confusable sound source, the corresponding spectrogram is input into the confusable sound source identification model for identification. The difference between the confusable sound source identification model and the artificial intelligence model is that only confusable sound sources are used for training and identification.

2. The method for identifying easily mixed sounds in an environment based on a double-layer model according to claim 1, characterized in that: Converting audio time series into spectrograms through Fourier transform with time window includes the following steps: Convert the mechanical vibration signal into an analog current signal through a microphone; Convert the continuous analog signal into a digital discrete time series signal through the audio collector; Slicing the discrete time series signal into audio time series vectors through a sliding time window, wherein the audio time series vectors have overlapping parts in the time domain; Perform Fourier transform on each audio time series vector and remove the phase angle result. Stack the audio time series results after Fourier transform in the direction of time order to obtain a spectrogram.

3. The method for identifying easily mixed sounds in an environment based on a double-layer model according to claim 1, wherein: The artificial intelligence model is used to implement the following steps: Slice the spectrum graph to obtain n subgraphs with overlapping parts; Reshape the subgraph into a one-dimensional array through linear transformation and add the sound source category feature vector with trainable parameters; The sub-graph position information vector is further merged into a one-dimensional array to obtain a feature matrix; The feature matrix is ​​input into the self-attention encoder to extract the pattern information in the spectrogram.

4. The method for identifying easily mixed sounds in an environment based on a double-layer model according to claim 3, characterized in that: The feature matrix is ​​input into the self-attention encoder to extract the pattern information in the spectrogram. The self-attention encoder includes L sub-encoder modules, and each sub-encoder module sequentially implements the following steps: Input the feature matrix into the first residual neural network and output the second feature matrix; The second feature matrix is ​​input into the second residual neural network, and the third feature matrix is ​​output. The third feature matrix is ​​input into the next sub-encoder module as the feature matrix; Among them, the first residual neural network and the second residual neural network each include a regularization neural network layer and a multi-head attention mechanism neural network layer.

5. The method for identifying easily mixed sounds in an environment based on a double-layer model according to claim 4, characterized in that: The multi-head attention mechanism neural network layer is implemented by the following steps: Execute the following steps h times in parallel: Use three linear transformation matrices to multiply the input feature matrix respectively to obtain the query Q, key value K and value V; Multiply the query Q by the key value K, divide it by the square root of the number of columns in the matrix, normalize it using the exponential function, and multiply the output by the value V; The results of h parallel executions are horizontally concatenated to obtain a new matrix, and the new matrix is ​​linearly transformed using a linear transformation matrix.

6. The method for identifying easily mixed sounds in an environment based on a double-layer model according to claim 4, characterized in that: The specific expression of the regularized neural network layer is: Where i = 1, 2, ..., N + 1, N represents the length of the sliding time window, Represents the sound source category feature vector of the i-th subgraph in the l-th sub-encoder module, E[] represents the calculation of mathematical expectation, Var[] represents the calculation of variance, ∈ is a very small amount to prevent the calculation of variance from being zero, resulting in the denominator being zero, γ and β represent the trainable parameters of the regular layer neural network.

7. The method for identifying easily mixed sounds in an environment based on a double-layer model according to claim 3, characterized in that: The size of the sub-image is set to 3×32×32.

8. A system for identifying easily mixed sounds in an environment based on a double-layer model, characterized in that: The system comprises: Spectrum acquisition module, used to convert audio time series into spectrogram through Fourier transform with time window; The two-layer model recognition module extracts image features from the spectrogram through an artificial intelligence model and identifies the sound source category. If the sound source category is an unconfusable sound source, the sound source recognition result is directly output. If the sound source category is a confusable sound source, the corresponding spectrogram is input into the confusable sound source recognition model for recognition. The difference between the confusable sound source recognition model and the artificial intelligence model is that only confusable sound sources are used for training and recognition.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps of the method for identifying environmental confusable sounds based on a double-layer model are implemented as claimed in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium having computer instructions stored thereon, characterized in that: When the instructions are executed by the processor, the steps of the method for identifying environmental confusable sounds based on a double-layer model are implemented as claimed in any one of claims 1 to 7.

Images