Sound event sample mixing method and device based on attention mechanism

Abstract

The present invention discloses a sound event sample mixing method and device based on an attention mechanism. In audio classification tasks, sample mixing is an effective data enhancement method. The original sample mixing scheme directly mixes the waveforms of two random samples, which not only ignores the temporal distribution of the sound event, but also may interfere with the original sound event in another sample. This paper proposes a sample mixing scheme based on an attention mechanism, which only selects those fragments containing sound events for mixing, rather than simply mixing the entire sample. The scheme uses the attention map of a pre-trained audio classification model to filter out the parts of the spectrum map that are useful for classification, and then selects the area for mixing. The scheme can significantly improve the accuracy of audio classification (+1.9mAP), especially the classification accuracy of short events (0.1s to 2s) can be increased by an average of 6.8%, while maintaining the classification accuracy of long events.

Description

Technical Field

[0001] The present invention relates to the field of digital media processing technology, and in particular to a method and device for mixing sound event samples based on an attention mechanism. Background Art

[0002] The audio classification task is to identify the sound events that occur in a given audio recording and enable various artificial intelligence-based systems to distinguish sounds and understand the acoustic environment. Its basic process is as follows Figure 1 Audio classification has a wide range of health and safety applications in homes, offices, industry, and transportation, and has become an active research topic in the field of acoustic signal processing.

[0003] State-of-the-art audio classification algorithms employ several data augmentation methods. These methods do not generate additional training data but instead directly modify input features. Mixup combines two audio samples in a specific ratio to create a mixed sample. Spectrogram masking masks a continuous range of frames or frequency bands with a specified value. Band augmentation involves strategically selecting frequency bands and applying amplification or subtraction to the signals in those bands.

[0004] However, existing techniques for processing entire spectrograms or entire frequency bands ignore the original distribution and duration of sound events. Blindly and randomly masking consecutive time frames or frequency bands can obscure information about the target event. Using the entire sample for mixing can interfere with sound events originally contained in another sample, thereby losing rather than adding information. This results in poor sample enhancement and a significant improvement in classification model accuracy. Summary of the Invention

[0005] This paper proposes a sample blending method based on an attention mechanism. This method selects only those segments containing sound events for blending, rather than blending the entire sample. This solves the problem of poor sample enhancement and insignificant improvement in classification accuracy in existing techniques that use the entire sample for blending.

[0006] In order to solve the above technical problems, the first aspect of the present invention provides a sound event sample mixing method based on an attention mechanism, comprising:

[0007] S1: Get the original audio samples and build a training dataset;

[0008] S2: Extract the logarithmic Mel spectrum for each audio sample in the training dataset;

[0009] S3: Input the logarithmic Mel-spectrogram of the audio sample into the pre-trained audio classification model based on the attention mechanism to classify the audio sample and obtain the attention map of the audio sample. The attention map includes multiple weights to describe the importance of each input feature to all other input features. The high and low weight values ​​indicate how much information the output feature obtains from the input feature at the corresponding position. Each input feature corresponds to a block.

[0010] S4: According to the weight values ​​in the attention map of the audio sample, select N blocks with the largest weight values ​​as anchor blocks, and use the center point of the anchor block as the anchor point, where N is a positive integer;

[0011] S5: Based on the selected anchor point and the pre-set threshold, the attention map is converted into a 0-1 mask map. The shape of the 0-1 mask map is consistent with the logarithmic Mel spectrum of the audio sample.

[0012] S6: Randomly select two audio samples from the training data set, and mix the two randomly selected audio samples according to the 0-1 masking map of one of the audio samples.

[0013] In one embodiment, before inputting the logarithmic Mel-spectrogram of the audio sample into the pre-trained attention-based audio classification model, the method further includes:

[0014] The log-Mel spectrum of the audio sample is divided into several 16×16 blocks, which are flattened and projected into features of a specific dimension.

[0015] In one embodiment, the pre-trained attention-based audio classification model is a Transformer model.

[0016] In one embodiment, step S5 includes:

[0017] S5.1: Initialize a 0-1 mask map with all positions set to 0 by default.

[0018] S5.2: Select an anchor block and set its center point (i, j) as the anchor point. The value of the center point (i, j) represents the signal amplitude value of the jth frequency band of the i-th frame;

[0019] S5.3: Based on the frame where the anchor point is located, select the 2W+1 frames near it and mark them as 1 on the 0-1 mask map. The range of the selected frames is [iW, i+W], where W is a preset parameter.

[0020] S5.4: Repeat steps S5.2 and S5.3 until all anchor blocks have generated corresponding regions and marked them on the 0-1 mask map.

[0021] In one embodiment, two audio samples are randomly selected from a training data set, their respective log mel spectra are x1 and x2, and their labels are y1 and y2. Step S6 mixes the two randomly selected audio samples according to a 0-1 masking map of one of the audio samples, including:

[0022] For each point (i, j) on the 0-1 masking map of the audio sample of x1, if the value of this point is 0, then That is, no blending is done; if the value of this point is 1, then That is, mixing is done, where the random number λ∈[0,1] represents the mixing ratio, which is sampled from the Beta distribution Beta(10,10). Represent the values ​​of the corresponding positions on the logarithmic Mel spectrum x2 of audio sample 2, the logarithmic Mel spectrum x1 of audio sample 1, and the mixed logarithmic Mel spectrum;

[0023] Generate the label of the mixed sample according to y1 and y2, y mix =λy1+(1-λ)y2,y mix is the label of the mixed sample.

[0024] Based on the same inventive concept, the second aspect of the present invention provides a sound event sample mixing device based on an attention mechanism, comprising:

[0025] The training set construction module is used to obtain original audio samples and construct a training data set;

[0026] The logarithmic Mel spectrum extraction module is used to extract the logarithmic Mel spectrum of each audio sample in the training data set;

[0027] The classification module is used to input the logarithmic Mel-spectrogram of the audio sample into the pre-trained audio classification model based on the attention mechanism to classify the audio sample and obtain the attention map of the audio sample. The attention map includes multiple weights to describe the importance of each input feature to all other input features. The high and low weight values ​​​​indicate how much information the output feature obtains from the input feature at the corresponding position. Each input feature corresponds to a block;

[0028] Anchor point selection module, which is used to select N blocks with the largest weight values ​​as anchor blocks based on the weight values ​​in the attention map of the audio sample, and use the center point of the anchor block as the anchor point, where N is a positive integer;

[0029] The mask map generation module is used to convert the attention map into a 0-1 mask map based on the selected anchor point and the pre-set threshold. The shape of the 0-1 mask map is consistent with the logarithmic Mel spectrum of the audio sample;

[0030] The mixing module is used to randomly select two audio samples from the training data set and mix the two randomly selected audio samples according to the 0-1 masking map of one of the audio samples.

[0031] Based on the same inventive concept, the third aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which implements the method described in the first aspect when the program is executed.

[0032] Based on the same inventive concept, the fourth aspect of the present invention provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in the first aspect when executing the program.

[0033] Compared with the prior art, the advantages and beneficial technical effects of the present invention are as follows:

[0034] The sample mixing method based on the attention mechanism provided by the present invention only selects those segments containing sound events for mixing, rather than simply mixing the entire sample. This solution uses a pre-trained audio classification model to obtain an attention map, selects the part of the spectrogram that is useful for classification (i.e., selects the anchor block), and then determines the area on the spectrogram that is ultimately used for mixing based on the selected anchor block, and then performs mixing. This solution can improve the effect of sample enhancement and significantly improve the accuracy of audio classification (+1.9mAP), especially the classification accuracy of short events (0.1s to 2s) can be increased by an average of 6.8%, while maintaining the classification accuracy of long events. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0036] Figure 1 The following is a basic flow chart of the sound event classification task in the prior art;

[0037] Figure 2 Flowchart of a sample mixing method based on an attention mechanism provided in an embodiment of the present invention;

[0038] Figure 3 This is an example diagram of selecting anchor blocks from a logarithmic Mel spectrum according to an embodiment of the present invention;

[0039] Figure 4 is a structural block diagram of a sample mixing device based on an attention mechanism in an embodiment of the present invention;

[0040] Figure 5 A schematic diagram of the structure of a computer-readable storage medium provided in an embodiment of the present invention;

[0041] Figure 6 A schematic diagram of the structure of a computer device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0042] The inventors of this application have discovered through extensive research and practice that the sample mixing scheme in the prior art directly mixes the waveforms of two random samples, which not only ignores the temporal distribution of sound events, but may also interfere with the original sound events in another sample. Based on this, a sample mixing scheme based on an attention mechanism is proposed, which only selects those fragments containing sound events for mixing, rather than simply mixing the entire sample. This scheme uses the attention map of a pre-trained audio classification model to filter out the parts of the spectrogram that are useful for classification, and then selects areas for mixing. This scheme can significantly improve the accuracy of audio classification (+1.9mAP), especially the classification accuracy of short events (0.1s to 2s) can be increased by an average of 6.8%, while maintaining the classification accuracy of long events.

[0043] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0044] Example 1

[0045] The embodiment of the present invention provides a sound event sample mixing method based on an attention mechanism, including:

[0046] S1: Get the original audio samples and build a training dataset;

[0047] S2: Extract the logarithmic Mel spectrum for each audio sample in the training dataset;

[0048] S3: Input the logarithmic Mel-spectrogram of the audio sample into the pre-trained audio classification model based on the attention mechanism to classify the audio sample and obtain the attention map of the audio sample. The attention map includes multiple weights to describe the importance of each input feature to all other input features. The high and low weight values ​​indicate how much information the output feature obtains from the input feature at the corresponding position. Each input feature corresponds to a block.

[0049] S4: According to the weight values ​​in the attention map of the audio sample, select N blocks with the largest weight values ​​as anchor blocks, and use the center point of the anchor block as the anchor point, where N is a positive integer;

[0050] S5: Based on the selected anchor point and the pre-set threshold, the attention map is converted into a 0-1 mask map. The shape of the 0-1 mask map is consistent with the logarithmic Mel spectrum of the audio sample.

[0051] S6: Randomly select two audio samples from the training data set, and mix the two randomly selected audio samples according to the 0-1 masking map of one of the audio samples.

[0052] The method of the present invention includes two main stages: a preprocessing stage and a training stage. The above steps S1 to S5 belong to the preprocessing stage, and step S6 belongs to the training stage.

[0053] The sample mixing method in the prior art (original sample mixing) and the sample mixing method of the present invention are described below:

[0054] Mixed methods analysis of the original sample

[0055] The original mixture combines two audio samples into a mixed sample in a certain ratio, and then mixes their labels in the same ratio, as shown in Formula 1:

[0056] x mix =λx1+(1-λ)x2

[0057] y mix =λy1+(1-λ)y2

[0058] Where x1 and x2 represent two randomly selected audio samples, and y1 and y2 represent the labels corresponding to the samples. λ∈[0, 1] is usually sampled from a Beta distribution. mix and y mix Represent the mixed samples and labels respectively.

[0059] However, simply mixing two random samples ignores the fact that the sound events in each sample are not evenly distributed across all frames. The duration of sound events of different categories varies greatly. For some short sound events, a considerable portion (5 to 8 seconds) of the entire 10-second sample is without a target event. To address this problem, the present invention proposes a sound event sample mixing method based on an attention mechanism, which only uses fragments that may contain sound events for mixing, rather than simply mixing two samples. This method reduces the interference caused by the original mixing method and improves the performance of models with different structures.

[0060] Analysis of the sample mixing method of the present invention

[0061] The method of the present invention does not directly mix the waveforms of the audio samples, but rather the log-mel spectrogram of the samples, because it not only clearly represents the time-frequency characteristics of the samples but is also the direct input of the model. The method obtains the possible locations of the sound events from the attention weights (attention map) of the pre-trained Transformer model, and then uses a pre-designed strategy to select areas on the spectrogram for mixing. The process is as follows: Figure 2 shown.

[0062] The horizontal axis of a logarithmic Mel-spectrogram represents time, and each column represents the signal strength of a frequency (e.g., 0-32000 Hz) within an audio frame (e.g., 100 ms). The location of a sound event refers to its start and end times (e.g., 2-3 seconds, corresponding to several columns in the spectrogram).

[0063] In one embodiment, before inputting the logarithmic Mel-spectrogram of the audio sample into the pre-trained attention-based audio classification model, the method further includes:

[0064] The log-Mel spectrum of the audio sample is divided into several 16×16 blocks, which are flattened and projected into features of a specific dimension.

[0065] In one embodiment, the pre-trained attention-based audio classification model is a Transformer model.

[0066] Specifically, the attention map output by the Transformer model is used to select anchor blocks.

[0067] This method obtains anchor information (anchor blocks) from the attention map of a pre-trained Transformer model on a certain audio sample. Similar to the common method in the field of image classification, the Transformer model for audio classification also divides the logarithmic Mel spectrum of the sample (which can be regarded as a single-channel image) into several 16×16 blocks, which are flattened and projected into features of a specific dimension. Then, a classification feature is attached to the head of the feature sequence (output feature sequence). The resulting sequence is fed into the Transformer model. In the output sequence of the encoder, only the classification feature at the head of the sequence is mapped to the target category through a linear layer. Therefore, by obtaining the attention weight of the classification feature on the entire sequence, we can infer the decision-making process of the model, that is, which blocks contain the information required for classification, and these blocks are the selected anchor blocks.

[0068] Specifically, in an N-layer Transformer model with M attention heads, there are N×M attention maps that describe the weight of the classification features for the entire sequence. The attention weight can be understood as how much information an output feature obtains from all the input features. For example, the input feature sequence has a total of 4 rows, corresponding to 4 features. The output feature has the same shape as the input and is a weighted representation of these 4 input features. The specific calculation method is standard matrix multiplication. The following uses the first input feature in the input feature sequence as an example to explain the meaning of the attention map. According to the rules of matrix multiplication, the first row of the attention map describes the weight of the first input feature for all other features (including itself). The higher the value of a certain weight, the more information the output feature obtains from the input feature at the corresponding position.

[0069] The Transformer model used in this example calculates an output feature sequence. Each feature in the output feature sequence undergoes the matrix calculation process described above, with only the top classification features used for the final classification. This is equivalent to summarizing the information contained in the classification features for the entire sample. After multiple rounds of training, the model will be able to extract the appropriate attention map for each sample, summarize the appropriate classification features, and ultimately output the correct classification result.

[0070] The obtained attention map is first normalized using min-max normalization, and then several blocks with the highest attention weights are selected as anchor blocks for generating a 0-1 mask map.

[0071] In a specific example, for an N-layer Transformer with M attention heads, N×M attention maps will be calculated for an audio sample. Each attention map is 1189×1189 in shape and describes the weight of each input vector (input feature sequence) to each output vector. We only take the first row of each attention map, which describes the weight of the first output feature (i.e., classification feature) in the output sequence to the 1189 input features, which means how much information the classification feature obtains from each input feature (i.e., each block). The larger the weight of a block, the more important this block is for correct classification. In this way, the "most important" blocks can be selected from the spectrogram and used as anchor blocks. See Figure 3 This is an example diagram of selecting anchor blocks from the logarithmic Mel spectrum in an embodiment of the present invention.

[0072] In one embodiment, step S5 includes:

[0073] S5.1: Initialize a 0-1 mask map with all positions set to 0 by default.

[0074] S5.2: Select an anchor block and set its center point (i, j) as the anchor point. The value of the center point (i, j) represents the signal amplitude value of the jth frequency band of the i-th frame;

[0075] S5.3: Based on the frame where the anchor point is located, select the 2W+1 frames near it and mark them as 1 on the 0-1 mask map. The range of the selected frames is [iW, i+W], where W is a preset parameter.

[0076] S5.4: Repeat steps S5.2 and S5.3 until all anchor blocks have generated corresponding regions and marked them on the 0-1 mask map.

[0077] Specifically, the selected anchor block is used to determine the region on the spectrogram that will ultimately be used for mixing. Since the time-frequency information of a sound event is always concentrated in adjacent frames and subbands, this method selects the region surrounding the anchor point for mixing and represents the selected region as a 0-1 mask map. Specifically, for a 16×16 anchor block, its center point is called the anchor point. This method determines the region for mixing by selecting the frame where the anchor point is located, as shown in steps S5.1 to S5.4.

[0078] In one embodiment, two audio samples are randomly selected from a training data set, their respective log mel spectra are x1 and x2, and their labels are y1 and y2. Step S6 mixes the two randomly selected audio samples according to a 0-1 masking map of one of the audio samples, including:

[0079] For each point (i, j) on the 0-1 masking map of the audio sample of x1, if the value of this point is 0, then That is, no blending is done; if the value of this point is 1, then That is, mixing is done, where the random number λ∈[0,1] represents the mixing ratio, which is sampled from the Beta distribution Beta(10,10). Represent the values ​​of the corresponding positions on the logarithmic Mel spectrum x2 of audio sample 2, the logarithmic Mel spectrum x1 of audio sample 1, and the mixed logarithmic Mel spectrum;

[0080] Generate the label of the mixed sample according to y1 and y2, y mix =λy1+(1-λ)y2,y mix is the label of the mixed sample.

[0081] The method proposed in the present invention is verified and illustrated through specific experiments below.

[0082] The proposed method operates in the sample enhancement phase of the sound event classification process. Three sample enhancement strategies—no mixing, raw mixing, and attention-based mixing—were employed to train three mainstream sound event classification models, and the average classification accuracy of the trained models was compared.

[0083] Table 1 Average classification accuracy of three mainstream models when trained with three different hybrid strategies

[0084]

[0085]

[0086] Table 1 shows the average accuracy results of three mainstream models when trained with three different hybrid strategies. The method of the present invention brings significant performance improvements on audio classification models of various structures.

[0087] Example 2

[0088] Based on the same inventive concept, this embodiment provides a sound event sample mixing device based on the attention mechanism, see Figure 4 , the device comprises:

[0089] The training set construction module 201 is used to obtain original audio samples and construct a training data set;

[0090] A logarithmic Mel spectrum extraction module 202 is used to extract a logarithmic Mel spectrum for each audio sample in the training data set;

[0091] A classification module 203 is configured to input the logarithmic Mel-spectrogram of the audio sample into a pre-trained audio classification model based on the attention mechanism to classify the audio sample and obtain an attention map of the audio sample. The attention map includes multiple weights to describe the importance of each input feature to all other input features. The high and low weight values ​​indicate how much information the output feature obtains from the input feature at the corresponding position. Each input feature corresponds to a block.

[0092] Anchor point selection module 204, configured to select N blocks with the largest weight values ​​as anchor blocks based on the weight values ​​in the attention map of the audio sample, and use the center point of the anchor block as the anchor point, where N is a positive integer;

[0093] a mask map generation module 205 for converting the attention map into a 0-1 mask map according to the selected anchor point and a preset threshold, wherein the shape of the 0-1 mask map is consistent with the logarithmic Mel spectrum of the audio sample;

[0094] The mixing module 206 is configured to randomly select two audio samples from the training data set and mix the two randomly selected audio samples according to the 0-1 masking map of one of the audio samples.

[0095] Since the device described in Example 2 of the present invention is the device used to implement the sound event sample mixing method based on the attention mechanism in Example 1 of the present invention, those skilled in the art will be able to understand the specific structure and variations of the device based on the method described in Example 1 of the present invention, and therefore will not be described in detail here. All devices used in the method in Example 1 of the present invention fall within the scope of protection of the present invention.

[0096] Example 3

[0097] Based on the same inventive concept, see Figure 5 The present invention further provides a computer-readable storage medium 300 on which a computer program 311 is stored. When the program is executed, the method described in the first embodiment is implemented.

[0098] Since the computer-readable storage medium described in Example 3 of the present invention is the computer-readable storage medium used to implement the sound event sample mixing method based on the attention mechanism in Example 1 of the present invention, those skilled in the art will be able to understand the specific structure and variations of the computer-readable storage medium based on the method described in Example 1 of the present invention, and therefore will not be described in detail here. All computer-readable storage media used in the method of Example 1 of the present invention fall within the scope of protection of the present invention.

[0099] Example 4

[0100] Based on the same inventive concept, the present application also provides a computer device, such as Figure 6 As shown, it includes a memory 401, a processor 402 and a computer program 403 stored in the memory and executable on the processor. When the processor executes the above program, the method in the first embodiment is implemented.

[0101] Since the computer device described in Example 4 of the present invention is the computer device used to implement the sound event sample mixing method based on the attention mechanism in Example 1 of the present invention, those skilled in the art will be able to understand the specific structure and variations of the computer device based on the method described in Example 1 of the present invention, and therefore will not be described in detail here. All computer devices used in the method in Example 1 of the present invention fall within the scope of protection of the present invention.

[0102] It will be understood by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0103] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0104] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0105] Obviously, those skilled in the art may make various changes and modifications to the embodiments of the present invention without departing from the spirit and scope of the embodiments of the present invention. Thus, if such changes and modifications of the embodiments of the present invention fall within the scope of the claims and their equivalents, the present invention is intended to include such changes and modifications.

Claims

1. A sound event sample mixing method based on the attention mechanism, characterized in that: include: S1: Get the original audio samples and build a training dataset; S2: Extract the logarithmic Mel spectrum for each audio sample in the training dataset; S3: Input the logarithmic Mel-spectrogram of the audio sample into the pre-trained audio classification model based on the attention mechanism to classify the audio sample and obtain the attention map of the audio sample. The attention map includes multiple weights to describe the importance of each input feature to all other input features. The high and low weight values ​​indicate how much information the output feature obtains from the input feature at the corresponding position. Each input feature corresponds to a block. S4: According to the weight values ​​in the attention map of the audio sample, select N blocks with the largest weight values ​​as anchor blocks, and use the center point of the anchor block as the anchor point, where N is a positive integer; S5: Based on the selected anchor point and the pre-set threshold, the attention map is converted into a 0-1 mask map. The shape of the 0-1 mask map is consistent with the logarithmic Mel spectrum of the audio sample. S6: Randomly select two audio samples from the training data set, and mix the two randomly selected audio samples according to the 0-1 masking map of one of the audio samples.

2. The sound event sample mixing method based on the attention mechanism according to claim 1, characterized in that: Before inputting the log-Mel spectrum of the audio sample into the pre-trained attention-based audio classification model, the method further includes: The log-Mel spectrum of the audio sample is divided into several 16×16 blocks, which are flattened and projected into features of a specific dimension.

3. The sound event sample mixing method based on the attention mechanism according to claim 2, characterized in that: The pre-trained attention-based audio classification model is the Transformer model.

4. The sound event sample mixing method based on the attention mechanism according to claim 1, characterized in that Step S5 includes: S5.1: Initialize a 0-1 mask map with all positions set to 0 by default. S5.2: Select an anchor block and set its center point (i, j) as the anchor point. The value of the center point (i, j) represents the signal amplitude value of the jth frequency band in the i-th frame. S5.3: Based on the frame where the anchor point is located, select the 2W+1 frames near it and mark them as 1 on the 0-1 mask map. The range of the selected frames is [iW,i+W], where W is a preset parameter. S5.4: Repeat steps S5.2 and S5.3 until all anchor blocks have generated corresponding regions and marked them on the 0-1 mask map.

5. The sound event sample mixing method based on the attention mechanism according to claim 1, characterized in that: Two audio samples are randomly selected from the training data set, their respective logarithmic Mel spectra are x1 and x2, and their labels are y1 and y2. Step S6 mixes the two randomly selected audio samples according to the 0-1 masking map of one of the audio samples, including: For each point (i, j) on the 0-1 masking map of the audio sample of x1, if the value of this point is 0, then That is, no blending is done; if the value of this point is 1, then That is, mixing is done, where the random number λ∈[0,1] represents the mixing ratio, which is sampled from the Beta distribution Beta(10,10). Represent the values ​​of the corresponding positions on the logarithmic Mel spectrum x2 of audio sample 2, the logarithmic Mel spectrum x1 of audio sample 1, and the mixed logarithmic Mel spectrum; Generate the label of the mixed sample according to y1 and y2, y min =λy1+(1-λ)y2,y mix is the label of the mixed sample.

6. A sound event sample mixing device based on an attention mechanism, characterized in that: include: The training set construction module is used to obtain original audio samples and construct a training data set; The logarithmic Mel spectrum extraction module is used to extract the logarithmic Mel spectrum of each audio sample in the training data set; The classification module is used to input the logarithmic Mel-spectrogram of the audio sample into the pre-trained audio classification model based on the attention mechanism to classify the audio sample and obtain the attention map of the audio sample. The attention map includes multiple weights to describe the importance of each input feature to all other input features. The high and low weight values ​​​​indicate how much information the output feature obtains from the input feature at the corresponding position. Each input feature corresponds to a block; Anchor point selection module, which is used to select N blocks with the largest weight values ​​as anchor blocks based on the weight values ​​in the attention map of the audio sample, and use the center point of the anchor block as the anchor point, where N is a positive integer; The mask map generation module is used to convert the attention map into a 0-1 mask map based on the selected anchor point and the pre-set threshold. The shape of the 0-1 mask map is consistent with the logarithmic Mel spectrum of the audio sample; The mixing module is used to randomly select two audio samples from the training data set and mix the two randomly selected audio samples according to the 0-1 masking map of one of the audio samples.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed, the method according to any one of claims 1 to 5 is implemented.

8. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the method according to any one of claims 1 to 5 is implemented.

Images