Audio recognition method, device, readable medium and electronic device

Abstract

The present disclosure relates to an audio recognition method, device, readable medium and electronic device, and relates to the field of electronic information processing technology. The method includes: obtaining audio to be recognized, extracting target audio features for characterizing the audio to be recognized, the dimension of the target audio features is positively correlated with the duration of the audio to be recognized, using a pre-trained recognition model, determining statistical features of a specified dimension based on the target audio features, and determining target audio events included in the audio to be recognized based on the statistical features, the recognition model is trained based on a pre-collected training audio set, and the training audio set includes training audio of different durations. The recognition model in the present disclosure extracts features from audio of different durations to obtain statistical features of a specified dimension, thereby recognizing audio events included in the audio, without the need to intercept or fill in the audio, and can retain complete information in the audio, thereby improving recognition accuracy.

Description

Technical Field

[0001] The present disclosure relates to the technical field of electronic information processing, and in particular, to an audio recognition method, device, readable medium, and electronic device. Background Art

[0002] With the continuous development of electronic information processing technology, audio, as an important medium for people to obtain information, has been widely used in daily life and work. To accurately and appropriately process audio, it is usually necessary to identify audio events contained in the audio, such as speaking events, singing events, and music events. When using machine learning to classify audio, the structure of the neural network often requires the input audio to be of a fixed duration. However, the duration of real audio varies. Therefore, the real audio needs to be truncated or padded, which can lead to information loss in the audio and reduce classification accuracy. Summary of the Invention

[0003] This section is provided to briefly introduce the concepts that will be described in detail in the detailed description below. This section is not intended to identify key features or essential features of the claimed technical solution, nor is it intended to limit the scope of the claimed technical solution.

[0004] In a first aspect, the present disclosure provides a method for recognizing audio, the method comprising:

[0005] Get the audio to be recognized;

[0006] Extracting a target audio feature for characterizing the audio to be recognized, wherein a dimension of the target audio feature is positively correlated with a duration of the audio to be recognized;

[0007] Using a pre-trained recognition model, the statistical features of the specified dimension are determined based on the target audio features, and the target audio events included in the audio to be recognized are determined based on the statistical features. The recognition model is trained based on a pre-collected training audio set, and the training audio set includes training audios of different lengths.

[0008] In a second aspect, the present disclosure provides an audio recognition device, the device comprising:

[0009] An acquisition module, used to acquire the audio to be recognized;

[0010] an extraction module, configured to extract target audio features for characterizing the audio to be recognized, wherein the dimension of the target audio features is positively correlated with the duration of the audio to be recognized;

[0011] The recognition module is used to use a pre-trained recognition model to determine the statistical features of a specified dimension based on the target audio features, and to determine the target audio events included in the audio to be recognized based on the statistical features. The recognition model is trained based on a pre-collected training audio set, and the training audio set includes training audio of different lengths.

[0012] In a third aspect, the present disclosure provides a computer-readable medium having a computer program stored thereon, which, when executed by a processing device, implements the steps of the method described in the first aspect of the present disclosure.

[0013] In a fourth aspect, the present disclosure provides an electronic device, comprising:

[0014] a storage device having a computer program stored thereon;

[0015] A processing device is used to execute the computer program in the storage device to implement the steps of the method described in the first aspect of the present disclosure.

[0016] Through the above technical solution, the present disclosure first obtains the audio to be recognized, and then extracts the target audio features used to characterize the audio to be recognized, wherein the dimension of the target audio features is positively correlated with the duration of the audio to be recognized. Finally, a pre-trained recognition model is used to determine the statistical features of the specified dimension based on the target audio features, and then the target audio events included in the audio to be recognized are determined based on the statistical features. The recognition model is trained using a pre-collected training audio set including training audios of different durations. The recognition model in the present disclosure extracts features from audios of different durations to obtain statistical features of the specified dimension, thereby identifying the audio events included in the audio. There is no need to intercept or fill in the audio, and the complete information in the audio can be retained, thereby improving the accuracy of recognition.

[0017] Other features and advantages of the present disclosure will be described in detail in the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The above and other features, advantages and aspects of the various embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that the originals and elements are not necessarily drawn to scale. In the drawings:

[0019] Figure 1 is a flowchart of an audio recognition method according to an exemplary embodiment;

[0020] Figure 2 is a flowchart of another audio recognition method according to an exemplary embodiment;

[0021] Figure 3 is a structural diagram of a recognition model according to an exemplary embodiment;

[0022] Figure 4 is a flowchart of a training recognition model according to an exemplary embodiment;

[0023] Figure 5 is a flowchart showing another method of training a recognition model according to an exemplary embodiment;

[0024] Figure 6 is a structural diagram of a recognition model and a duration statistical model according to an exemplary embodiment;

[0025] Figure 7 is a block diagram of an audio recognition device according to an exemplary embodiment;

[0026] Figure 8 is a block diagram of another audio recognition device according to an exemplary embodiment;

[0027] Figure 9 It is a block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0028] The following describes embodiments of the present disclosure in more detail with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments described herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are for illustrative purposes only and are not intended to limit the scope of protection of the present disclosure.

[0029] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0030] As used herein, the term "including" and its variations are open-ended, i.e., "including but not limited to." The term "based on" means "based, at least in part, on." The term "one embodiment" means "at least one embodiment," the term "another embodiment" means "at least one additional embodiment," and the term "some embodiments" means "at least some embodiments." Other terms are defined in the following description.

[0031] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0032] It should be noted that the modifications of "one" and "multiple" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, they should be understood as "one or more".

[0033] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0034] Figure 1 FIG. 1 is a flow chart of an audio recognition method according to an exemplary embodiment. Figure 1 As shown, the method includes the following steps:

[0035] Step 101: Acquire audio to be recognized.

[0036] Step 102: extract target audio features for characterizing the audio to be recognized, where the dimension of the target audio features is positively correlated with the duration of the audio to be recognized.

[0037] For example, first, the audio to be recognized can be obtained. The audio to be recognized can be locally stored audio, audio obtained from a server via a network, or audio recorded by a user in real time. This disclosure does not specifically limit this. The duration of the audio to be recognized can be any length. Before recognizing the audio to be recognized, the audio to be recognized can be pre-processed, for example, by performing noise reduction processing on the audio to be recognized.

[0038] Afterwards, the target audio features that can characterize the audio to be identified can be extracted, wherein the dimension of the target audio features is positively correlated with the duration of the audio to be identified, that is, the longer the duration of the audio to be identified, the larger the dimension of the target audio features, and the shorter the duration of the audio to be identified, the smaller the dimension of the target audio features. Specifically, the target audio features may include a time dimension and a frequency dimension, and the time dimension of the target audio features is positively correlated with the duration of the audio to be identified, and the frequency dimension may remain unchanged. It can be understood that the audio to be identified includes multiple audio frames, and accordingly, the target audio features may include features that can characterize each audio frame. Therefore, the longer the duration of the audio to be identified and the greater the number of audio frames, the larger the time dimension of the target audio features.

[0039] The target audio feature can be, for example, a Mel-spectrogram feature. Methods for extracting the Mel-spectrogram feature of the audio to be recognized can include performing pre-emphasis, framing, windowing, fast Fourier transform, Mel-filtering, logarithmization, and normalization on the audio to be recognized to obtain the Mel-spectrogram feature. Mel-filtering can be implemented, for example, using an 80-dimensional Mel-log filter bank. If the duration of the audio to be recognized is 5 seconds, consisting of 400 audio frames, the frequency dimension of the Mel-spectrogram feature of the audio to be recognized is 80, and the time dimension of the Mel-spectrogram feature is 400. Therefore, the Mel-spectrogram feature can be a 400*80-dimensional vector.

[0040] Step 103: Using a pre-trained recognition model, the statistical features of the specified dimension are determined based on the target audio features, and the target audio events included in the audio to be recognized are determined based on the statistical features. The recognition model is trained based on a pre-collected training audio set, and the training audio set includes training audios of different lengths.

[0041] For example, the target audio features can be used as input to identify the audio to be identified using a pre-trained recognition model to determine the target audio events included in the audio to be identified. The recognition model is trained using a pre-collected training audio set, which includes multiple training audios. The duration of the multiple training audios can be the same or different, that is, the training audios in the training audio set can be of any duration.

[0042] The recognition model can first determine the statistical features of a specified dimension based on the target audio features. The statistical features can characterize the statistics of the features of multiple audio frames in the audio to be recognized, and the dimension of the statistical features is fixed, that is, the specified dimension. Afterwards, the recognition model can use a classifier to classify the statistical features of the specified dimension to determine the target audio event included in the audio to be recognized. It can be understood that the recognition model can classify the audio to be recognized. Specifically, the classifier can determine the matching degree of the statistical features with each audio event in a plurality of pre-specified audio events, that is, the higher the matching degree of the statistical features with the audio event, the higher the probability value of including the audio event in the audio to be recognized, and the lower the matching degree, the lower the probability value of including the audio event in the audio to be recognized. The recognition model can determine the audio event with the highest corresponding matching degree as the target audio event, or it can determine the audio event whose matching degree meets the preset conditions (for example, arranged in descending order, with a preset number of matching degrees at the front) as the target audio event, that is, the target audio event can be one or more, and the present disclosure does not make specific limitations on this. The various pre-specified audio events may be, for example, speaking events, singing events, music events, doorbell events, etc., or may be human voice events, background sound events, animal sound events, etc.

[0043] If features with non-fixed dimensions are used to classify the audio to be recognized, the input of the classifier in the recognition model will be uncertain, that is, the structure of the classifier is uncertain, and therefore the classification task cannot be completed. In the present disclosure, the dimensions of the statistical features are fixed (that is, they are all specified dimensions). When classifying the audio to be recognized, the input of the classifier in the recognition model is fixed, so the audio to be recognized can be classified. Furthermore, since the recognition model can recognize audio to be recognized of any length, there is no need to intercept or fill in the audio to be recognized, so the recognition model can learn the complete information in the audio to be recognized, thereby improving the accuracy of recognition.

[0044] In summary, the present disclosure first obtains the audio to be recognized, and then extracts the target audio features used to characterize the audio to be recognized, wherein the dimension of the target audio features is positively correlated with the duration of the audio to be recognized. Finally, a pre-trained recognition model is used to determine the statistical features of a specified dimension based on the target audio features, and then the target audio events included in the audio to be recognized are determined based on the statistical features. The recognition model is trained using a pre-collected training audio set including training audios of different durations. The recognition model in the present disclosure extracts features from audios of different durations to obtain statistical features of a specified dimension, thereby identifying the audio events included in the audio. There is no need to intercept or fill in the audio, and the complete information in the audio can be retained, thereby improving the accuracy of recognition.

[0045] Figure 2 is a flow chart showing another audio recognition method according to an exemplary embodiment. Figure 2 As shown, the structure of the recognition model can be as follows Figure 3 As shown, it includes: multiple convolutional layers, statistical pooling layers and classification layers connected in sequence.

[0046] Accordingly, step 103 can be implemented by the following steps:

[0047] In step 1031, the target audio features are input into the first convolutional layer to obtain high-level features of the audio to be recognized output by the last convolutional layer. The dimension of the high-level features is positively correlated with the dimension of the target audio features.

[0048] First, the target audio features can be input into the first convolutional layer, and the input of each subsequent convolutional layer is the output of the previous convolutional layer, until the output of the last convolutional layer is obtained, which is the high-level features of the audio to be identified, wherein the dimension of the high-level features is positively correlated with the dimension of the target audio features, that is, the dimension of the high-level features is positively correlated with the duration of the audio to be identified. In other words, the dimension of the high-level features is variable. The longer the duration of the audio to be identified, the larger the dimension of the high-level features, and the shorter the duration of the audio to be identified, the smaller the dimension of the high-level features. It can be understood that the output of the first convolutional layer is the low-level features of the audio to be identified, and after multiple subsequent convolutional layers, the high-level features of the audio to be identified can be extracted.

[0049] The structure of the convolutional layer can be, for example, a CNN (Convolutional Neural Networks), and the convolution kernel can be, for example, 3*3. Furthermore, the convolutional layer can also include a dropout layer (Dropout) and a pooling layer (Pooling). The dropout layer can reduce the overfitting problem of the convolutional neural network, and the pooling layer can reduce the size of the features and reduce the amount of calculation. For example, a maximum pooling layer with a kernel size of 2*2 can be selected. In this way, the convolutional neural network + dropout layer + pooling layer can be combined into one convolutional layer. For example, the target audio feature is a 400*80-dimensional vector. After being processed by the convolutional neural network in the first convolutional layer, a 400*80*32-dimensional vector can be output. After passing through the dropout layer and pooling layer in the first convolutional layer, a 200*40*32-dimensional vector can be output. The 200*40*32-dimensional vector can then be input into the next convolutional layer, and so on, until the last convolutional layer reduces the second dimension in the vector to 1, obtaining a T*1*256-dimensional vector, that is, a T*256-dimensional vector, where T is positively correlated with the duration of the audio to be recognized, that is, positively correlated with the dimension of the target audio feature.

[0050] Step 1032: input the high-level features into the statistical pooling layer to obtain statistical features output by the statistical pooling layer.

[0051] Step 1033: Input the statistical features into the classification layer to determine the target audio event according to the output of the classification layer.

[0052] For example, after obtaining the high-level features output by the last convolutional layer, the high-level features can be input into the statistical pooling layer, and the statistical pooling layer can output the statistics of the high-level features, i.e., statistical features. The statistics can be one or more, and correspondingly, the statistical features can be the result of splicing one or more statistics, that is, the dimension of the statistical features is determined according to the type of statistics. For example, the statistic can be the average value, or the average value and variance, or the average value, variance and median, etc., and the present disclosure does not make specific limitations on this. The statistical pooling layer can extract more statistical information based on the high-level features, that is, the recognition model can learn more information. At the same time, the statistical pooling layer can receive input of variable dimensions (i.e., high-level features of different dimensions) and output statistical features of fixed dimensions.

[0053] Finally, the statistical features are input into the classification layer to determine the target audio event based on the output of the classification layer. The classification layer can be composed of multiple fully connected layers connected in sequence. The statistical features are input into the first fully connected layer, and the output of the last fully connected layer is applied to the softmax function to obtain the matching degree of each audio event with multiple pre-specified audio events. The audio event with the highest matching degree is then determined as the target audio event.

[0054] In one implementation, the statistical pooling layer includes multiple statistical modules, each of which corresponds to a statistical process. Step 1032 may include:

[0055] Step 1) Input high-level features into each statistical module respectively to obtain statistical results output by the statistical module, which are the results of the corresponding statistical processing performed by the statistical module on the high-level features.

[0056] Step 2) The statistical results output by multiple statistical modules are spliced ​​to obtain statistical features, and the specified dimension is determined according to the number of statistical modules.

[0057] Specifically, the statistical pooling layer can include multiple statistical modules, each of which corresponds to a statistical process. That is, each statistical module is used to perform a statistical process on high-level features and calculate a statistical quantity (i.e., statistical result) of the high-level features. The statistical results output by each statistical module can then be spliced ​​to obtain statistical features. The specified dimension is determined by the number of statistical modules.

[0058] For example, the statistical pooling layer includes an average statistics module and a variance statistics module. The average statistics module corresponds to average processing, and the variance statistics module corresponds to variance processing. If the high-level feature is a T*256-dimensional vector, then the high-level feature is input into the average statistics module. The average statistics module can calculate the average value of T 1*256-dimensional vectors to obtain a 1*256-dimensional average value vector (i.e., statistical result). At the same time, the high-level feature is input into the variance statistics module. The variance statistics module can calculate the variance of T 1*256-dimensional vectors to obtain a 1*256-dimensional variance vector (i.e., statistical result). Finally, the average value vector and the variance vector can be spliced ​​through concat to obtain a 2*256-dimensional statistical feature, where 2*256 is the specified dimension.

[0059] Figure 4 FIG. 1 is a flow chart showing a training recognition model according to an exemplary embodiment. Figure 4 As shown in Figure 2, the recognition model is trained in the following way:

[0060] Step A: Obtain a sample input set and a sample output set. The sample input set includes multiple sample inputs, and the sample input includes a training audio in the training audio set. The sample output set includes sample outputs corresponding to each sample input, and each sample output includes a real audio event included in the corresponding training audio.

[0061] In step B, the sample input set is used as the input of the recognition model, and the sample output set is used as the output of the recognition model to train the recognition model.

[0062] For example, when training a recognition model, it is necessary to first obtain a sample input set. The sample input set includes multiple sample inputs, and the sample input can be a training audio in the training audio set. The duration of each sample input can be the same or different, that is, each sample input can be of any duration. After obtaining the sample input set, a sample output set can be further obtained. The sample output set includes sample outputs corresponding to each sample input, and each sample output includes a real audio event included in the corresponding training audio, wherein the real audio event can be one or more, and the present disclosure does not specifically limit this.

[0063] When training the recognition model, the sample input set can be used as the input of the recognition model, and the sample output set can be used as the output of the recognition model to train the recognition model so that when the sample input set is input, the output of the recognition model can match the sample output set. For example, the cross entropy loss can be determined based on the output of the recognition model and the sample output set. With the goal of reducing the cross entropy loss, the back propagation algorithm is used to correct the neuron parameters in the recognition model. The neuron parameters can be, for example, the weight (English: Weight) and bias (English: Bias) of the neuron. Repeat the above steps until the cross entropy loss meets the preset conditions, for example, the cross entropy loss is less than the preset loss threshold, so as to achieve the purpose of training the recognition model. Specifically, in the process of training the recognition model, the learning rate can be set to 0.0001, the optimizer can be selected as: Adam, the batch size can be set to: 64, and the epoch can be set to: 100.

[0064] Generally, when using machine learning to classify audio, the structure of the neural network often requires that the duration of the input audio be fixed. Accordingly, when training the neural network, the duration of the training audio is also required to be fixed. During the test process, multiple groups of classification scores are usually obtained through sliding window operations, and then the classification scores of different windows are averaged to obtain the final classification result. Sliding window operations and averaging processing will cause the scores of some audio events with shorter durations to be always low, reducing the accuracy of the trained neural network. In the training process of the recognition model disclosed in the present invention, the training audio set includes training audios of different durations, which can support training of training audios of different durations, avoid sliding window operations and averaging processing during the test process, solve the problem of low recognition accuracy of audio events with shorter durations, and improve the accuracy of the recognition model.

[0065] Figure 5 is a flowchart of another training recognition model according to an exemplary embodiment. Figure 5 As shown, the implementation of step B may include:

[0066] Step B1: Input any sample in the current batch as the input of the recognition model to obtain the output of the recognition model.

[0067] Step B2: Determine similar sample inputs in the current batch that match the sample input through the duration statistical model, and determine the statistical duration of all similar sample inputs. The sample outputs corresponding to similar sample inputs are the same as the sample outputs corresponding to the sample input.

[0068] Step B3: Determine the recognition loss based on the output of the recognition model, the sample output corresponding to the sample input, and the statistical duration.

[0069] Step B4: With the goal of reducing recognition loss, the recognition model is trained using the back propagation algorithm.

[0070] For example, among audio events, there are often audio events with shorter duration, such as doorbell events, dog barking events, etc. Since the duration of such audio events is shorter, they have less impact on the training process of the recognition model, and are likely to reduce the accuracy of the recognition model in identifying such audio events. Therefore, when determining the loss of the recognition model, a duration statistical model can be set to determine the statistical duration of each sample input, and the loss of the recognition model can be determined based on the statistical duration to train the recognition model. The connection between the duration statistical model and the recognition model is as follows: Figure 6 shown.

[0071] Specifically, any sample input in the current batch can be used as the input of the recognition model to obtain the output of the recognition model. Then, the duration statistical model is used to determine the similar sample inputs that match the sample input in the current batch, and the statistical duration of all similar sample inputs is determined, wherein the sample output corresponding to the similar sample input is the same as the sample output corresponding to the sample input. The statistical duration can be understood as the duration of the real audio event corresponding to the current sample input in the batch. For example, the sample output corresponding to a certain sample input is a doorbell event, and there are 5 sample inputs in the current batch whose corresponding sample outputs are doorbell events, and the corresponding durations are 1s, 1.5s, 1.2s, 0.5s, and 0.9s, respectively. Then the statistical duration is 1+1.5+1.2+0.5+0.9=5.1s.

[0072] The recognition loss can then be determined based on the output of the recognition model, the sample output corresponding to the sample input, and the statistical duration. Finally, with the goal of reducing the recognition loss, the recognition model is trained using the backpropagation algorithm.

[0073] In one implementation, step B3 may be implemented by the following steps:

[0074] Step 3) According to the ratio of the statistical duration to the total duration, determine the duration coefficient corresponding to the sample input. The duration coefficient is negatively correlated with the ratio. The total duration is the sum of the durations of each sample input in the sample input set.

[0075] Step 4) Determine the recognition loss based on the output of the recognition model, the product of the sample output corresponding to the sample input and the duration coefficient.

[0076] For example, the duration coefficient corresponding to the sample input can be determined based on the ratio of the statistical duration to the total duration. The total duration is the sum of the durations of each sample input in the sample input set, that is, the total duration of the training audio set. The duration coefficient is negatively correlated with the ratio. For example, duration coefficient = total duration / statistical duration. For example, if the total duration is 120s and the statistical duration is 5.1s, then the duration coefficient = 120 / 5.1 = 24.5.

[0077] Finally, the recognition loss can be determined by multiplying the output of the recognition model, the sample output corresponding to the sample input, and the duration coefficient. For example, the recognition loss can be determined by the following formula:

[0078]

[0079] Among them, L represents the recognition loss, N represents the number of sample inputs in the current batch, M represents the number of types of audio events, and t c Indicates that the corresponding sample output in the current batch is the duration coefficient corresponding to the sample input of the c-th audio event, y i,c Indicates whether the i-th sample input in the current batch includes the c-th audio event, which is 1 if included and 0 if not. i,c Represents the output of the recognition model corresponding to the i-th sample input in the current batch.

[0080] In this way, by adjusting the recognition loss through the duration coefficient, the influence of shorter audio events on the recognition loss is increased, and the influence of longer audio events on the recognition loss is reduced, thereby reducing the influence of the duration of different audio events on the recognition model training process and improving the accuracy of the recognition model in recognizing shorter audio events.

[0081] In summary, the present disclosure first obtains the audio to be recognized, and then extracts the target audio features used to characterize the audio to be recognized, wherein the dimension of the target audio features is positively correlated with the duration of the audio to be recognized. Finally, a pre-trained recognition model is used to determine the statistical features of a specified dimension based on the target audio features, and then the target audio events included in the audio to be recognized are determined based on the statistical features. The recognition model is trained using a pre-collected training audio set including training audios of different durations. The recognition model in the present disclosure extracts features from audios of different durations to obtain statistical features of a specified dimension, thereby identifying the audio events included in the audio. There is no need to intercept or fill in the audio, and the complete information in the audio can be retained, thereby improving the accuracy of recognition.

[0082] Figure 7 is a block diagram of an audio recognition device according to an exemplary embodiment. Figure 7 As shown, the apparatus 200 may include:

[0083] The acquisition module 201 is used to acquire the audio to be recognized.

[0084] The extraction module 202 is configured to extract target audio features for characterizing the audio to be recognized, wherein the dimension of the target audio features is positively correlated with the duration of the audio to be recognized.

[0085] The recognition module 203 is used to use a pre-trained recognition model to determine the statistical features of a specified dimension based on the target audio features, and to determine the target audio events included in the audio to be recognized based on the statistical features. The recognition model is trained based on a pre-collected training audio set, and the training audio set includes training audio of different lengths.

[0086] Figure 8 is a block diagram of another audio recognition device according to an exemplary embodiment. Figure 8 As shown in the figure, the recognition model includes: multiple convolutional layers, statistical pooling layers and classification layers connected in sequence.

[0087] The identification module 203 may include:

[0088] The convolution submodule 2031 is used to input the target audio features into the first convolution layer to obtain high-level features of the audio to be recognized output by the last convolution layer, where the dimensions of the high-level features are positively correlated with the dimensions of the target audio features.

[0089] The statistical pooling submodule 2032 is used to input high-level features into the statistical pooling layer to obtain statistical features output by the statistical pooling layer.

[0090] The classification submodule 2033 is configured to input the statistical features into the classification layer to determine the target audio event according to the output of the classification layer.

[0091] In one implementation, the statistical pooling layer includes multiple statistical modules, each statistical module corresponding to a statistical processing.

[0092] Accordingly, the statistical pooling submodule 2032 can be used to perform the following steps:

[0093] Step 1) Input high-level features into each statistical module respectively to obtain statistical results output by the statistical module, which are the results of the corresponding statistical processing performed by the statistical module on the high-level features.

[0094] Step 2) The statistical results output by multiple statistical modules are spliced ​​to obtain statistical features, and the specified dimension is determined according to the number of statistical modules.

[0095] In another implementation, the recognition model is trained by:

[0096] Step A: Obtain a sample input set and a sample output set. The sample input set includes multiple sample inputs, and the sample input includes a training audio in the training audio set. The sample output set includes sample outputs corresponding to each sample input, and each sample output includes a real audio event included in the corresponding training audio.

[0097] In step B, the sample input set is used as the input of the recognition model, and the sample output set is used as the output of the recognition model to train the recognition model.

[0098] In yet another implementation, step B may include:

[0099] Step B1: Input any sample in the current batch as the input of the recognition model to obtain the output of the recognition model.

[0100] Step B2: Determine similar sample inputs in the current batch that match the sample input through the duration statistical model, and determine the statistical duration of all similar sample inputs. The sample outputs corresponding to similar sample inputs are the same as the sample outputs corresponding to the sample input.

[0101] Step B3: Determine the recognition loss based on the output of the recognition model, the sample output corresponding to the sample input, and the statistical duration.

[0102] Step B4: With the goal of reducing recognition loss, the recognition model is trained using the back propagation algorithm.

[0103] Specifically, step B3 can be implemented by the following steps:

[0104] Step 3) According to the ratio of the statistical duration to the total duration, determine the duration coefficient corresponding to the sample input. The duration coefficient is negatively correlated with the ratio. The total duration is the sum of the durations of each sample input in the sample input set.

[0105] Step 4) Determine the recognition loss based on the output of the recognition model, the product of the sample output corresponding to the sample input and the duration coefficient.

[0106] Regarding the apparatus in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0107] In summary, the present disclosure first obtains the audio to be recognized, and then extracts the target audio features used to characterize the audio to be recognized, wherein the dimension of the target audio features is positively correlated with the duration of the audio to be recognized. Finally, a pre-trained recognition model is used to determine the statistical features of a specified dimension based on the target audio features, and then the target audio events included in the audio to be recognized are determined based on the statistical features. The recognition model is trained using a pre-collected training audio set including training audios of different durations. The recognition model in the present disclosure extracts features from audios of different durations to obtain statistical features of a specified dimension, thereby identifying the audio events included in the audio. There is no need to intercept or fill in the audio, and the complete information in the audio can be retained, thereby improving the accuracy of recognition.

[0108] Reference below Figure 9 , which shows a schematic structural diagram of an electronic device (e.g., an execution subject of the embodiments of the present disclosure, such as a terminal device or a server) 300 suitable for implementing the embodiments of the present disclosure. The terminal device in the embodiments of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 9 The electronic device shown is only an example and should not limit the functions and scope of use of the embodiments of the present disclosure.

[0109] like Figure 9 As shown, the electronic device 300 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 301, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 302 or a program loaded from a storage device 308 into a random access memory (RAM) 303. Various programs and data required for the operation of the electronic device 300 are also stored in the RAM 303. The processing device 301, the ROM 302, and the RAM 303 are connected to each other via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0110] Typically, the following devices may be connected to the I / O interface 305: an input device 306 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 307 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 308 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 309. The communication device 309 may allow the electronic device 300 to communicate with other devices wirelessly or by wire to exchange data. Figure 9The electronic device 300 is shown with various devices, but it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed instead.

[0111] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 309, or installed from the storage device 308, or installed from the ROM 302. When the computer program is executed by the processing device 301, the above-mentioned functions defined in the method of the embodiment of the present disclosure are performed.

[0112] It should be noted that the computer-readable medium mentioned above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, device, or component. In the present disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0113] In some embodiments, the terminal devices and servers may communicate using any currently known or future developed network protocol, such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0114] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0115] The above-mentioned computer-readable medium carries one or more programs. When the above-mentioned one or more programs are executed by the electronic device, the electronic device is enabled to: obtain the audio to be recognized; extract the target audio features used to characterize the audio to be recognized, and the dimension of the target audio features is positively correlated with the duration of the audio to be recognized; use a pre-trained recognition model to determine the statistical features of a specified dimension based on the target audio features, and determine the target audio events included in the audio to be recognized based on the statistical features, and the recognition model is trained based on a pre-collected training audio set, and the training audio set includes training audios of different durations.

[0116] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, or a combination thereof, including, but not limited to, object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on the remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0117] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0118] The modules described in the embodiments of the present disclosure may be implemented in software or hardware. In some cases, the name of a module does not necessarily limit the module itself. For example, an acquisition module may also be described as a "module for acquiring audio to be recognized."

[0119] The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, and without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.

[0120] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0121] According to one or more embodiments of the present disclosure, Example 1 provides an audio recognition method, including: obtaining audio to be recognized; extracting target audio features for characterizing the audio to be recognized, the dimension of the target audio features being positively correlated with the duration of the audio to be recognized; using a pre-trained recognition model, determining statistical features of specified dimensions based on the target audio features, and determining target audio events included in the audio to be recognized based on the statistical features, the recognition model being trained based on a pre-collected training audio set, and the training audio set including training audios of different durations.

[0122] According to one or more embodiments of the present disclosure, Example 2 provides the method of Example 1, wherein the recognition model includes: multiple convolutional layers, statistical pooling layers, and classification layers connected in sequence; the pre-trained recognition model is used to determine the statistical features of a specified dimension based on the target audio features, and the target audio events included in the audio to be recognized are determined based on the statistical features, including: inputting the target audio features into the first convolutional layer to obtain high-level features of the audio to be recognized output by the last convolutional layer, and the dimensions of the high-level features are positively correlated with the dimensions of the target audio features; inputting the high-level features into the statistical pooling layer to obtain the statistical features output by the statistical pooling layer; and inputting the statistical features into the classification layer to determine the target audio event based on the output of the classification layer.

[0123] According to one or more embodiments of the present disclosure, Example 3 provides the method of Example 2, wherein the statistical pooling layer includes multiple statistical modules, each statistical module corresponding to a statistical processing; the high-level features are input into the statistical pooling layer to obtain the statistical features output by the statistical pooling layer, including: inputting the high-level features into each of the statistical modules respectively to obtain the statistical results output by the statistical module, the statistical results being the results of the corresponding statistical processing performed by the statistical module on the high-level features; the statistical results output by multiple statistical modules are spliced ​​to obtain the statistical features, and the specified dimension is determined according to the number of the statistical modules.

[0124] According to one or more embodiments of the present disclosure, Example 4 provides the methods of Examples 1 to 3, and the recognition model is trained in the following manner: obtaining a sample input set and a sample output set, the sample input set including multiple sample inputs, the sample input including one of the training audios in the training audio set, the sample output set including sample outputs corresponding to each of the sample inputs, and each of the sample outputs including a real audio event included in the corresponding training audio; using the sample input set as the input of the recognition model and the sample output set as the output of the recognition model to train the recognition model.

[0125] According to one or more embodiments of the present disclosure, Example 5 provides the method of Example 4, which uses the sample input set as the input of the recognition model and the sample output set as the output of the recognition model to train the recognition model, including: using any sample input in the current batch as the input of the recognition model to obtain the output of the recognition model; determining similar sample inputs that match the sample input in the current batch through a time statistical model, and determining the statistical time of all similar sample inputs, and the sample outputs corresponding to the similar sample inputs are the same as the sample outputs corresponding to the sample input; determining the recognition loss based on the output of the recognition model, the sample output corresponding to the sample input, and the statistical time; and training the recognition model using a backpropagation algorithm with the goal of reducing the recognition loss.

[0126] According to one or more embodiments of the present disclosure, Example 6 provides the method of Example 5, which determines the recognition loss based on the output of the recognition model, the sample output corresponding to the sample input, and the statistical duration, including: determining the duration coefficient corresponding to the sample input based on the ratio of the statistical duration and the total duration, the duration coefficient is negatively correlated with the ratio, and the total duration is the sum of the durations of each sample input in the sample input set; determining the recognition loss based on the output of the recognition model, the sample output corresponding to the sample input, and the product of the duration coefficient.

[0127] According to one or more embodiments of the present disclosure, Example 7 provides an audio recognition device, including: an acquisition module for acquiring audio to be recognized; an extraction module for extracting target audio features for characterizing the audio to be recognized, wherein the dimension of the target audio features is positively correlated with the duration of the audio to be recognized; a recognition module for using a pre-trained recognition model to determine statistical features of specified dimensions based on the target audio features, and determine target audio events included in the audio to be recognized based on the statistical features, wherein the recognition model is trained based on a pre-collected training audio set, and the training audio set includes training audios of different durations.

[0128] According to one or more embodiments of the present disclosure, Example 8 provides the device of Example 7, wherein the recognition model includes: multiple convolutional layers, statistical pooling layers and classification layers connected in sequence; the recognition module includes: a convolution submodule, used to input the target audio features into the first convolutional layer to obtain the high-level features of the audio to be recognized output by the last convolutional layer, and the dimensions of the high-level features are positively correlated with the dimensions of the target audio features; a statistical pooling submodule, used to input the high-level features into the statistical pooling layer to obtain the statistical features output by the statistical pooling layer; and a classification submodule, used to input the statistical features into the classification layer to determine the target audio event based on the output of the classification layer.

[0129] According to one or more embodiments of the present disclosure, Example 9 provides the device of Example 8, wherein the statistical pooling layer includes multiple statistical modules, each statistical module corresponds to a statistical processing; the statistical pooling submodule is used to: input the high-level features into each of the statistical modules respectively to obtain the statistical results output by the statistical module, and the statistical results are the results of the corresponding statistical processing performed by the statistical module on the high-level features; the statistical results output by multiple statistical modules are spliced ​​to obtain the statistical features, and the specified dimension is determined according to the number of the statistical modules.

[0130] According to one or more embodiments of the present disclosure, Example 10 provides the apparatus of Examples 7-9, wherein the recognition model is trained in the following manner: obtaining a sample input set and a sample output set, wherein the sample input set includes multiple sample inputs, the sample input includes one of the training audios in the training audio set, and the sample output set includes sample outputs corresponding to each of the sample inputs, and each of the sample outputs includes a real audio event included in the corresponding training audio; using the sample input set as the input of the recognition model and the sample output set as the output of the recognition model to train the recognition model.

[0131] According to one or more embodiments of the present disclosure, Example 11 provides the apparatus of Example 10, wherein the sample input set is used as the input of the recognition model, and the sample output set is used as the output of the recognition model to train the recognition model, including: using any of the sample inputs in the current batch as the input of the recognition model to obtain the output of the recognition model; determining similar sample inputs in the current batch that match the sample input through a time statistical model, and determining the statistical time of all the similar sample inputs, and the sample outputs corresponding to the similar sample inputs are the same as the sample outputs corresponding to the sample inputs; determining the recognition loss based on the output of the recognition model, the sample output corresponding to the sample input, and the statistical time; and training the recognition model using a backpropagation algorithm with the goal of reducing the recognition loss.

[0132] According to one or more embodiments of the present disclosure, Example 12 provides the apparatus of Example 11, which determines the recognition loss based on the output of the recognition model, the sample output corresponding to the sample input, and the statistical duration, including: determining the duration coefficient corresponding to the sample input based on the ratio of the statistical duration and the total duration, the duration coefficient is negatively correlated with the ratio, and the total duration is the sum of the durations of each sample input in the sample input set; determining the recognition loss based on the output of the recognition model, the sample output corresponding to the sample input, and the product of the duration coefficient.

[0133] According to one or more embodiments of the present disclosure, Example 13 provides a computer-readable medium having a computer program stored thereon, which implements the steps of the methods described in Examples 1 to 6 when executed by a processing device.

[0134] According to one or more embodiments of the present disclosure, Example 14 provides an electronic device, comprising: a storage device on which a computer program is stored; and a processing device for executing the computer program in the storage device to implement the steps of the method described in Examples 1 to 6.

[0135] The above description is merely a preferred embodiment of the present disclosure and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by the specific combination of the above-mentioned technical features, but also includes other technical solutions formed by any combination of the above-mentioned technical features or their equivalents without departing from the above-mentioned disclosed concepts. For example, a technical solution formed by replacing the above-mentioned features with (but not limited to) technical features with similar functions disclosed in this disclosure.

[0136] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details have been included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0137] Although the subject matter has been described using language specific to structural features and / or methodological logical acts, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are merely example forms of implementing the claims. Regarding the apparatus in the above-described embodiments, the specific manner in which each module performs operations has been described in detail in the embodiments related to the method and will not be elaborated upon here.

Claims

1. A method for audio recognition, characterized in that: The method comprises: Get the audio to be recognized; Extracting a target audio feature for characterizing the audio to be recognized, wherein a dimension of the target audio feature is positively correlated with a duration of the audio to be recognized; Utilizing a pre-trained recognition model, determining statistical features of a specified dimension based on the target audio features, and determining target audio events included in the audio to be recognized based on the statistical features, wherein the recognition model is trained based on a pre-collected training audio set, wherein the training audio set includes training audio of varying durations; The recognition model is trained in the following way: Acquire a sample input set and a sample output set, wherein the sample input set includes multiple sample inputs, each sample input includes one of the training audios in the training audio set, and the sample output set includes sample outputs corresponding to each of the sample inputs, each of the sample outputs includes a real audio event included in the corresponding training audio; Inputting any of the samples in the current batch as input to the recognition model to obtain an output of the recognition model; Using a duration statistical model, determine similar sample inputs in the current batch that match the sample input, and determine the statistical duration of all similar sample inputs. The sample outputs corresponding to the similar sample inputs are the same as the sample output corresponding to the sample input. Determining recognition loss based on the output of the recognition model, the sample output corresponding to the sample input, and the statistical duration; With the goal of reducing the recognition loss, the recognition model is trained using a back-propagation algorithm.

2. The method according to claim 1, characterized in that The recognition model includes: a plurality of convolutional layers, statistical pooling layers, and classification layers connected in sequence; the use of the pre-trained recognition model to determine statistical features of a specified dimension based on the target audio features, and determining target audio events included in the audio to be recognized based on the statistical features, including: Inputting the target audio feature into the first convolutional layer to obtain high-level features of the audio to be recognized output by the last convolutional layer, where the dimension of the high-level features is positively correlated with the dimension of the target audio feature; Inputting the high-level features into the statistical pooling layer to obtain the statistical features output by the statistical pooling layer; The statistical features are input into the classification layer to determine the target audio event according to an output of the classification layer.

3. The method according to claim 2, characterized in that The statistical pooling layer includes multiple statistical modules, each statistical module corresponds to a statistical processing; Inputting the high-level features into the statistical pooling layer to obtain the statistical features output by the statistical pooling layer includes: Inputting the high-level features into each of the statistical modules respectively to obtain statistical results output by the statistical module, wherein the statistical results are the results of the statistical module performing corresponding statistical processing on the high-level features; The statistical results output by multiple statistical modules are spliced ​​to obtain the statistical feature, and the specified dimension is determined according to the number of the statistical modules.

4. The method according to claim 1, wherein The determining of the recognition loss according to the output of the recognition model, the sample output corresponding to the sample input, and the statistical duration includes: Determine a duration coefficient corresponding to the sample input according to the ratio of the statistical duration to the total duration, where the duration coefficient is negatively correlated with the ratio, and the total duration is the sum of the durations of each sample input in the sample input set; The recognition loss is determined according to the product of the output of the recognition model, the sample output corresponding to the sample input, and the duration coefficient.

5. An audio recognition device, characterized in that: The device comprises: An acquisition module, used to acquire the audio to be recognized; an extraction module, configured to extract target audio features for characterizing the audio to be recognized, wherein the dimension of the target audio features is positively correlated with the duration of the audio to be recognized; a recognition module, configured to use a pre-trained recognition model to determine statistical features of a specified dimension based on the target audio features, and to determine target audio events included in the audio to be recognized based on the statistical features, wherein the recognition model is trained based on a pre-collected training audio set, wherein the training audio set includes training audio of different durations; The recognition model is trained in the following way: Acquire a sample input set and a sample output set, wherein the sample input set includes multiple sample inputs, each sample input includes one of the training audios in the training audio set, and the sample output set includes sample outputs corresponding to each of the sample inputs, each of the sample outputs includes a real audio event included in the corresponding training audio; Inputting any of the samples in the current batch as input to the recognition model to obtain an output of the recognition model; Using a duration statistical model, determine similar sample inputs in the current batch that match the sample input, and determine the statistical duration of all similar sample inputs. The sample outputs corresponding to the similar sample inputs are the same as the sample output corresponding to the sample input. Determining recognition loss based on the output of the recognition model, the sample output corresponding to the sample input, and the statistical duration; With the goal of reducing the recognition loss, the recognition model is trained using a back-propagation algorithm.

6. The device according to claim 5, characterized in that The recognition model includes: a plurality of convolutional layers, statistical pooling layers and classification layers connected in sequence; The identification module includes: a convolution submodule, configured to input the target audio feature into the first convolution layer to obtain high-level features of the audio to be recognized output by the last convolution layer, wherein the dimension of the high-level features is positively correlated with the dimension of the target audio feature; A statistical pooling submodule, configured to input the high-level features into the statistical pooling layer to obtain the statistical features output by the statistical pooling layer; The classification submodule is configured to input the statistical features into the classification layer to determine the target audio event according to an output of the classification layer.

7. A computer-readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processing device, the steps of the method according to any one of claims 1 to 4 are implemented.

8. An electronic device, characterized in that: include: a storage device having one or more computer programs stored thereon; One or more processing devices, configured to execute the one or more computer programs in the storage device to implement the steps of the method according to any one of claims 1 to 4.

Images