Voice recognition method, voice recognition device and electronic equipment

Abstract

The present disclosure provides a sound recognition method, a sound recognition device and an electronic device. The method includes obtaining audio information of a user object whose age is within a target age range, wherein the user object expresses needs through sound characteristics; preprocessing the audio information to obtain target audio information; extracting features of the target audio information to obtain logarithmic Mel spectrum features; processing the logarithmic Mel spectrum features according to a preset signal processing rule to obtain an initial two-dimensional matrix; processing the initial two-dimensional matrix using a preset difference processing rule to obtain a target two-dimensional matrix; generating a three-dimensional tensor according to the target two-dimensional matrix and the initial two-dimensional matrix; and inputting the three-dimensional tensor into a classifier recognition module to output a prediction result, wherein the prediction result is used to characterize the needs of the user object.

Description

Technical Field

[0001] The present disclosure relates to the technical field of speech signal processing, and more specifically, to a sound recognition method, a sound recognition device, an electronic device, a computer-readable storage medium, and a computer program product. Background Art

[0002] Sound is a sound wave generated by the vibration of an object. It is a wave phenomenon that is transmitted through a medium (air or solid, liquid) and can be sensed by the auditory organs of humans or animals. Different needs can be expressed through sound. For example, a baby can make a cry to remind relatives of its needs, such as hunger, cold, etc.

[0003] In the process of realizing the concept of the present disclosure, the inventors found that there are at least the following problems in the related art: it is difficult to judge the needs of the user through the characteristics of the user's voice. Summary of the invention

[0004] In view of this, embodiments of the present disclosure provide a sound recognition method, a sound recognition device, an electronic device, a computer-readable storage medium, and a computer program product.

[0005] One aspect of an embodiment of the present disclosure provides a sound recognition method, including:

[0006] Acquiring audio information of user objects whose ages are within a target age group, wherein the user objects express their needs through voice characteristics;

[0007] Preprocessing the above audio information to obtain target audio information;

[0008] Perform feature extraction on the target audio information to obtain logarithmic Mel spectrum features;

[0009] The above-mentioned logarithmic Mel spectrum features are processed according to a preset signal processing rule to obtain an initial two-dimensional matrix;

[0010] Processing the initial two-dimensional matrix using a preset difference processing rule to obtain a target two-dimensional matrix; generating a three-dimensional tensor according to the target two-dimensional matrix and the initial two-dimensional matrix; and

[0011] The three-dimensional tensor is input into a classifier recognition module, and a prediction result is output, wherein the prediction result is used to characterize the needs of the user object.

[0012] According to an embodiment of the present disclosure, the above-mentioned initial two-dimensional matrix includes a target frequency dimension and a target time dimension;

[0013] The above-mentioned initial two-dimensional matrix is ​​processed by using a preset difference processing rule to obtain a target two-dimensional matrix, including:

[0014] The initial two-dimensional matrix is ​​subjected to difference processing in the target frequency dimension and the target time dimension respectively to obtain the target two-dimensional matrix.

[0015] According to an embodiment of the present disclosure, the above-mentioned preprocessing of the audio information to obtain the target audio information includes:

[0016] Downsampling the sampling rate of the audio information to obtain first audio information;

[0017] Performing noise reduction processing on the first audio information to obtain second audio information;

[0018] The second audio information is subjected to silence suppression processing to obtain the target audio information.

[0019] According to an embodiment of the present disclosure, performing silence suppression processing on the second audio information to obtain the target audio information includes:

[0020] The silent segment in the second audio information is processed using a voice activity detection method to obtain the target audio information.

[0021] According to an embodiment of the present disclosure, the above-mentioned classifier recognition module includes a sound recognition model, and the above-mentioned sound recognition model includes a feature embedding layer, a model encoder and a multi-layer perceptron head;

[0022] The above-mentioned three-dimensional tensor is input into the classifier recognition module, and the prediction result is output, including:

[0023] Cut the three-dimensional tensor according to a preset cutting rule to obtain multiple sub-tensors;

[0024] Input the plurality of the sub-tensors into the feature embedding layer, and output a first matrix;

[0025] Splicing the preset vectors on the first matrix to obtain a second matrix;

[0026] According to the second matrix and the preset position coding matrix, a third matrix is ​​obtained;

[0027] Input the third matrix into the model encoder and output a fourth matrix;

[0028] The fourth matrix is ​​input into the multi-layer perceptron head, and the prediction result is output.

[0029] According to an embodiment of the present disclosure, the step of inputting the third matrix into the model encoder and outputting the fourth matrix includes:

[0030] Input the third matrix into the model encoder and output a new third matrix;

[0031] When the preset condition is not satisfied, iteratively using the new third matrix as the third matrix to input the third matrix into the model encoder;

[0032] When the preset conditions are met, the new third matrix is ​​determined as the fourth matrix.

[0033] According to an embodiment of the present disclosure, the above-mentioned model encoder includes a first normalization layer, a multi-head attention mechanism layer, a second normalization layer and a multi-layer perceptron;

[0034] The step of inputting the third matrix into the model encoder and outputting the fourth matrix comprises:

[0035] Input the third matrix into the first normalization layer to obtain a normalized third matrix;

[0036] The third matrix after the normalization is processed by a linear layer to obtain a target feature matrix;

[0037] According to the above target feature matrix, three three-dimensional tensors are obtained;

[0038] Input the above three 3D tensors into the above multi-head attention mechanism layer and output the fifth matrix;

[0039] Generate a sixth matrix according to the fifth matrix and the third matrix;

[0040] Input the sixth matrix into the second normalization layer, and output a seventh matrix;

[0041] Input the seventh matrix into the multilayer perceptron and output an eighth matrix;

[0042] The fourth matrix is ​​generated according to the eighth matrix and the sixth matrix.

[0043] According to an embodiment of the present disclosure, the above-mentioned logarithmic Mel spectrum feature includes a time dimension and a frequency dimension;

[0044] The above-mentioned logarithmic Mel spectrum features are processed according to the preset signal processing rules to obtain an initial two-dimensional matrix, including:

[0045] Processing the above frequency dimension according to the preset frequency condition to obtain the target frequency dimension;

[0046] The above time dimension is processed according to the preset time condition to obtain the target time dimension;

[0047] The initial two-dimensional matrix is ​​constructed according to the target frequency dimension and the target time dimension.

[0048] Another aspect of the present disclosure provides a voice recognition device, including:

[0049] An acquisition module, used to acquire audio information of user objects whose ages are within a target age group, wherein the user objects express their needs through voice characteristics;

[0050] A preprocessing module, used to preprocess the above audio information to obtain target audio information;

[0051] A feature extraction module is used to extract features of the target audio information to obtain logarithmic Mel spectrum features;

[0052] A first processing module is used to process the logarithmic Mel spectrum feature according to a preset signal processing rule to obtain an initial two-dimensional matrix;

[0053] A second processing module, used for processing the initial two-dimensional matrix using a preset difference processing rule to obtain a target two-dimensional matrix;

[0054] A generating module, used for generating a three-dimensional tensor according to the target two-dimensional matrix and the initial two-dimensional matrix; and

[0055] The prediction module is used to input the three-dimensional tensor into the classifier recognition module and output a prediction result, wherein the prediction result is used to characterize the needs of the user object.

[0056] Another aspect of an embodiment of the present disclosure provides an electronic device, comprising: one or more processors; and a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method described above.

[0057] Another aspect of an embodiment of the present disclosure provides a computer-readable storage medium storing computer-executable instructions, which are used to implement the method described above when executed.

[0058] Another aspect of an embodiment of the present disclosure provides a computer program product, wherein the computer program product includes computer executable instructions, and the instructions are used to implement the method described above when executed.

[0059] According to an embodiment of the present disclosure, the above-mentioned logarithmic Mel spectrum features are processed by using preset signal processing rules to obtain an initial two-dimensional matrix, and then the obtained initial two-dimensional matrix is ​​processed by using preset difference processing rules, and the two-dimensional matrix obtained after the difference processing is combined with the initial two-dimensional matrix to generate a three-dimensional tensor, and the three-dimensional tensor is input into a classifier recognition module to predict user needs. This technical means at least partially overcomes the technical problem that it is difficult to judge user needs through user voice characteristics, thereby improving the accuracy of identifying user needs through voice characteristics. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] The above and other objects, features and advantages of the present disclosure will become more apparent through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, in which:

[0061] Figure 1 An exemplary system architecture to which the sound recognition method according to an embodiment of the present disclosure can be applied is schematically shown;

[0062] Figure 2 The flowchart of the voice recognition method according to the embodiment of the present disclosure is schematically shown;

[0063] Figure 3 A flowchart for obtaining prediction results according to an embodiment of the present disclosure is schematically shown;

[0064] Figure 4 The structure diagram of the voice recognition method according to the embodiment of the present disclosure is schematically shown;

[0065] Figure 5 A confusion matrix diagram of the UAR indicator according to an embodiment of the present disclosure is schematically shown;

[0066] Figure 6 A block diagram schematically shows a voice recognition device according to an embodiment of the present disclosure; and

[0067] Figure 7 The block diagram schematically shows an electronic device for implementing a voice recognition method according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0068] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present disclosure. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0069] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0070] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification, and should not be interpreted in an idealized or overly rigid manner.

[0071] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0072] Crying is the main way for babies to communicate with the outside world. According to surveys, nearly 130 million babies are born every year in the world. Experienced doctors, nurses and parents can understand their thoughts and needs through their crying. However, for new parents, it is very difficult to judge their thoughts and needs through the crying of babies. In addition, some diseases can also be predicted through the crying of babies. Because the crying of an unhealthy baby contains some characteristics that are different from those of a healthy baby. If parents can understand and judge accurately, they can give timely treatment to the baby. In the prior art, spectrograms or logarithmic Mel spectrograms are usually used as input features of the classification model to predict the needs of babies.

[0073] However, the cry signal of a baby is mainly composed of two segments, exhalation and inhalation, and compared with the voice of an adult, the frequency of the cry signal of a baby is mainly distributed between 1500Hz and 3000Hz. Using a spectrogram or a logarithmic Mel spectrum to represent the cry of a baby will result in a significant loss of information.

[0074] People with language barriers cannot express their needs through precise language, but can only express their needs through limited and vague sounds. It is also difficult to judge the needs of such people through their sounds.

[0075] In view of this, the embodiments of the present disclosure provide a sound recognition method, a sound recognition device, an electronic device, a computer-readable storage medium, and a computer program product. The method includes obtaining audio information of a user object whose age is within a target age range, wherein the user object expresses needs through sound characteristics; preprocessing the audio information to obtain target audio information; extracting features of the target audio information to obtain logarithmic Mel spectrum features; processing the logarithmic Mel spectrum features according to preset signal processing rules to obtain an initial two-dimensional matrix; processing the initial two-dimensional matrix using preset differential processing rules to obtain a target two-dimensional matrix; generating a three-dimensional tensor based on the target two-dimensional matrix and the initial two-dimensional matrix; and inputting the three-dimensional tensor into a classifier recognition module to output a prediction result, wherein the prediction result is used to characterize the needs of the user object.

[0076] Figure 1 The exemplary system architecture 100 to which the sound recognition method according to the embodiment of the present disclosure can be applied is schematically shown. It should be noted that: Figure 1 What is shown is merely an example of a system architecture to which the embodiments of the present disclosure can be applied, in order to help those skilled in the art understand the technical content of the present disclosure, but it does not mean that the embodiments of the present disclosure cannot be used in other devices, systems, environments or scenarios.

[0077] like Figure 1 As shown, the system architecture 100 according to this embodiment may include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired and / or wireless communication links, etc.

[0078] Users can use terminal devices 101, 102, 103 to interact with server 105 through network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, 103, such as audio collection applications, audio transmission applications, search applications, instant messaging tools, email clients and / or social platform software, etc. (only as examples).

[0079] The terminal devices 101 , 102 , and 103 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, and desktop computers.

[0080] The server 105 may be a server that provides various services, such as a background management server (only as an example) that provides support for websites browsed by users using the terminal devices 101, 102, and 103. The background management server may analyze and process the received data such as user requests, and feed back the processing results (such as prediction results, information, or data obtained or generated according to the user request) to the terminal device.

[0081] It should be noted that the sound recognition method provided in the embodiment of the present disclosure can generally be executed by the server 105. Accordingly, the sound recognition device provided in the embodiment of the present disclosure can generally be set in the server 105. The sound recognition method provided in the embodiment of the present disclosure can also be executed by a server or server cluster that is different from the server 105 and can communicate with the terminal devices 101, 102, 103 and / or the server 105. Accordingly, the sound recognition device provided in the embodiment of the present disclosure can also be set in a server or server cluster that is different from the server 105 and can communicate with the terminal devices 101, 102, 103 and / or the server 105. Alternatively, the sound recognition method provided in the embodiment of the present disclosure can also be executed by the terminal devices 101, 102, or 103, or can also be executed by other terminal devices different from the terminal devices 101, 102, or 103. Accordingly, the sound recognition device provided in the embodiment of the present disclosure can also be set in the terminal devices 101, 102, or 103, or can be set in other terminal devices different from the terminal devices 101, 102, or 103.

[0082] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0083] Figure 2 The flowchart of the voice recognition method according to the embodiment of the present disclosure is schematically shown.

[0084] like Figure 2 As shown, the voice recognition method may include operations S201 to S207.

[0085] In operation S201, audio information of a user object whose age is within a target age range is obtained, wherein the user object expresses a demand through voice characteristics.

[0086] In operation S202, the audio information is preprocessed to obtain target audio information.

[0087] In operation S203, feature extraction is performed on the target audio information to obtain logarithmic Mel frequency spectrum features.

[0088] In operation S204, the logarithmic Mel-spectrum features are processed according to a preset signal processing rule to obtain an initial two-dimensional matrix.

[0089] In operation S205, the initial two-dimensional matrix is ​​processed using a preset difference processing rule to obtain a target two-dimensional matrix.

[0090] In operation S206 , a three-dimensional tensor is generated according to the target two-dimensional matrix and the initial two-dimensional matrix.

[0091] In operation S207, the three-dimensional tensor is input into a classifier recognition module, and a prediction result is output, wherein the prediction result is used to characterize the needs of the user object.

[0092] According to the embodiments of the present disclosure, user objects may refer to people in the target age group, such as infants aged 0 to 3 years old, or special people with language barriers. In order to facilitate the description of the present application, the following embodiments are illustrated by infants aged 0 to 3 years old and special people with language barriers.

[0093] According to an embodiment of the present disclosure, for infants aged 0 to 3 years old, their audio information may include crying sounds, and the infants may express the following needs through crying sounds: "peed", "hungry", "sleepy", "uncomfortable", "painful" and "need a hug", etc.

[0094] According to an embodiment of the present disclosure, for a special group of people with language barriers, their audio information may include humming sounds, and they may express the following needs through humming sounds of different types and tones: "I'm hungry", "uncomfortable" and "pain", etc.

[0095] According to an embodiment of the present disclosure, the logarithmic Mel spectrum feature may refer to a logarithmic spectrum feature represented based on a nonlinear Mel scale.

[0096] According to an embodiment of the present disclosure, the preset signal processing rules may include downsampling, noise reduction and other processing of audio information. The preset difference processing rules may refer to performing primary difference and secondary difference processing on the initial two-dimensional matrix. The classifier recognition module includes a sound recognition model for identifying and classifying three-dimensional tensors to facilitate output of prediction results under different classifications, wherein the sound recognition model may include a Vision Transformer model.

[0097] According to an embodiment of the present disclosure, after the audio information of infants or special groups is collected by a device, the audio information may be preprocessed to obtain target audio information with better quality because there are some factors in the audio information that affect the judgment of the needs of infants or special groups. Logarithmic Mel spectrum features may be obtained after feature extraction of the target audio information. An initial two-dimensional matrix may be constructed by processing the logarithmic Mel spectrum features, and a target two-dimensional matrix may be obtained by differential processing of the initial two-dimensional matrix, which is then combined with the initial two-dimensional matrix to construct a three-dimensional tensor. The three-dimensional tensor is input into a classifier recognition module to output a prediction result that can characterize the needs of infants.

[0098] According to an embodiment of the present disclosure, the above-obtained initial two-dimensional matrix is ​​processed by utilizing preset differential processing rules, and the two-dimensional matrix obtained after differential processing is combined with the initial two-dimensional matrix to generate a three-dimensional tensor, and the three-dimensional tensor is input into a classifier recognition module to predict user needs. This technical means at least partially overcomes the technical problem that it is difficult to judge user needs through the user's voice characteristics, thereby improving the accuracy of identifying user needs through voice characteristics.

[0099] According to an embodiment of the present disclosure, the initial two-dimensional matrix includes a target frequency dimension and a target time dimension.

[0100] According to an embodiment of the present disclosure, processing an initial two-dimensional matrix using a preset difference processing rule to obtain a target two-dimensional matrix may include the following operations:

[0101] The initial two-dimensional matrix is ​​subjected to difference processing in the target frequency dimension and the target time dimension respectively to obtain the target two-dimensional matrix.

[0102] According to an embodiment of the present disclosure, the differential processing may include a primary differential processing and a secondary differential processing.

[0103] According to an embodiment of the present disclosure, when the logarithmic Mel spectrum feature is extracted, the logarithmic Mel spectrum feature is processed using a preset signal processing rule to obtain an initial two-dimensional matrix X. For example, the initial two-dimensional matrix may be a (224, 224) matrix, where 224 may refer to a feature vector. The initial two-dimensional matrix is ​​subjected to a primary difference processing and a secondary difference processing in the target frequency dimension and the target time dimension, respectively, to obtain ΔTX and Δ F The target two-dimensional matrix of X, both of which can also be (224, 224) matrices.

[0104] According to an embodiment of the present disclosure, the calculation of the differential processing is shown in formula (1) and formula (2).

[0105]

[0106]

[0107] Among them, f represents the frequency of the initial two-dimensional matrix; t represents the frequency of the initial two-dimensional matrix; n represents a point in the initial two-dimensional matrix corresponding to the target audio information, where n can be 1 or 2; N represents the differential window of length 2 used when extracting features of the target audio information, where N can be 2.

[0108] According to the embodiment of the present disclosure, according to the obtained Δ T X and Δ F X and the initial two-dimensional matrix X can construct at least one of the following three-dimensional tensors: (X, Δ T X, ), (X, Δ T X, ), (X, Δ F X, ), (X, Δ F X, and

[0109] (X, Δ T X, Δ F X).

[0110] Among them, T can represent the target time dimension, and F can represent the target frequency dimension.

[0111] According to an embodiment of the present disclosure, preprocessing the audio information to obtain the target audio information may include the following operations:

[0112] The sampling rate of the audio information is downsampled to obtain the first audio information. The first audio information is subjected to noise reduction to obtain the second audio information. The second audio information is subjected to silence suppression to obtain the target audio information.

[0113] According to an embodiment of the present disclosure, performing silence suppression processing on the second audio information to obtain the target audio information may include the following operations:

[0114] The silent segment in the second audio information is processed using a voice activity detection method to obtain target audio information.

[0115] According to an embodiment of the present disclosure, the frequency of the baby crying signal is mainly distributed between 1500 Hz and 3000 Hz, and the sampling rate of the baby audio information collected by the collection device includes 44.1 kHz or 32 kHz.

[0116] According to an embodiment of the present disclosure, after collecting audio information of infants or special groups, for example, the sampling rate can be downsampled from a higher frequency to a lower frequency, for example, the audio data of 44.1kHz or 32kHz can be downsampled to a sampling rate of 16kHz, thereby obtaining the first audio information. In order to remove low-frequency environmental noise and other interfering sounds, a high-pass filter can be used to filter the first audio information to obtain the second audio information.

[0117] According to the embodiments of the present disclosure, due to the intermittent nature of infant crying and the vocalization of special groups, there are silent audio segments in the second audio information, and the silent audio segments can be removed by using the Voice Activity Detection (VAD) method. Finally, feature extraction is performed on the target audio information processed by the Voice Activity Detection method.

[0118] According to an embodiment of the present disclosure, when performing feature extraction, a 32 millisecond Hanning window can be used, a frame shift of 16 milliseconds, and the number of Mel filters is 224 to obtain a logarithmic Mel spectrum feature. Then, the number of time frames is 224, the insufficient ones are filled with 0, and the excess ones are cut off according to the feature processing rule, thereby obtaining an initial two-dimensional matrix X.

[0119] According to an embodiment of the present disclosure, the classifier recognition module includes a sound recognition model, and the sound recognition model includes a feature embedding layer, a model encoder and a multi-layer perceptron head.

[0120] Figure 3 The flowchart of obtaining prediction results according to an embodiment of the present disclosure is schematically shown.

[0121] Figure 4 The structure diagram of the voice recognition method according to the embodiment of the present disclosure is schematically shown.

[0122] like Figure 3 As shown, the three-dimensional tensor is input into the classifier recognition module, and the prediction result is output, including operations S301 to S306.

[0123] In operation S301, a three-dimensional tensor is cut according to a preset cutting rule to obtain a plurality of sub-tensors.

[0124] In operation S302, a plurality of sub-tensors are input into a feature embedding layer, and a first matrix is ​​output.

[0125] In operation S303, the preset vectors are concatenated on the first matrix to obtain a second matrix.

[0126] In operation S304, a third matrix is ​​obtained according to the second matrix and a preset position coding matrix.

[0127] In operation S305, the third matrix is ​​input to the model encoder, and a fourth matrix is ​​output.

[0128] In operation S306, the fourth matrix is ​​input into the multi-layer perceptron head, and a prediction result is output.

[0129] According to an embodiment of the present disclosure, in the three-dimensional tensor Y=(X, ΔTX, Δ F X), Y∈R F×T×C , where R is the real number field, C can be 3, which represents the number of channel dimensions of the features, that is, the three feature matrices formed.

[0130] like Figure 4 As shown, the three-dimensional tensor Y is divided into a preset number of sub-tensors, for example, 196 sub-tensors of size (16, 16, 3). The sub-tensors are input into the feature embedding layer and then converted into YP∈R 196 × 768 A preset vector is concatenated on the output first matrix YP to obtain a second matrix of (197, 768), where the preset vector can be a class token vector of (1, 768). A preset position encoding matrix (Position Enbedding) of the same shape and size is added to the second matrix to obtain a third matrix h0, where the third matrix h0 is shown in formula (3).

[0131]

[0132] Among them, E∈R 768×768 , E pos ∈R 197×768 .

[0133] like Figure 4 As shown, the third matrix h0 is input into the model encoder, and the fourth matrix is ​​output. Then, the multilayer perceptron head processes the fourth matrix to obtain the prediction result.

[0134] According to an embodiment of the present disclosure, inputting the third matrix into a model encoder and outputting the fourth matrix may include the following operations:

[0135] The third matrix is ​​input into the model encoder, and a new third matrix is ​​output. If the preset condition is not met, the new third matrix is ​​iteratively used as the third matrix to input the third matrix into the model encoder. If the preset condition is met, the new third matrix is ​​determined as the fourth matrix.

[0136] like Figure 4As shown, in order to ensure the prediction accuracy of the classifier recognition module, multiple iterations can be performed when the third matrix is ​​input into the model encoder module, and iterative calculations are performed when the preset conditions are not met. Among them, the preset condition can be the number of iterations L, and its specific value can be specifically set according to actual calculation requirements, for example, L can be 12 times.

[0137] According to an embodiment of the present disclosure, the model encoder may include a first normalization layer, a multi-head attention mechanism layer, a second normalization layer and a multi-layer perceptron.

[0138] like Figure 4 As shown, the third matrix is ​​input into the model encoder, and the fourth matrix is ​​output, which may include the following operations:

[0139] The third matrix is ​​input into the first normalization layer to obtain the normalized third matrix. The normalized third matrix is ​​processed by the linear layer to obtain the target feature matrix. According to the target feature matrix, three three-dimensional tensors are obtained. The three three-dimensional tensors are input into the multi-head attention mechanism layer to output the fifth matrix.

[0140] The sixth matrix is ​​generated based on the fifth matrix and the third matrix. The sixth matrix is ​​input into the second normalization layer, and the seventh matrix is ​​output. The seventh matrix is ​​input into the multilayer perceptron, and the eighth matrix is ​​output. The fourth matrix is ​​generated based on the eighth matrix and the sixth matrix.

[0141] According to an embodiment of the present disclosure, when the third matrix h0 is input into the first normalization layer, the first normalization layer may perform normalization processing on the third matrix h0 to obtain the normalized third matrix.

[0142] The normalized calculation is shown in formula (4).

[0143]

[0144] Among them, x represents the value in h0, y represents the normalized value of a certain numerical point, E[x] represents the mean of the data in h0, Var[x] represents the variance of the data in h0, ∈ ensures that the square root is positive, γ is the gain, and β is the deviation.

[0145] According to an embodiment of the present disclosure, a linear layer is used to perform linear processing on the third matrix after normalization to obtain a target feature matrix. According to the target feature matrix, three three-dimensional tensors are obtained, for example, three tensors with shapes of [12, 197, 64], where 12 represents the number of heads of the multi-head attention mechanism, and [197, 64] represents the size of the Q, K, and V matrices, that is, each tensor contains 12 Q, K, and V matrices, respectively, where Q, K, and V represent query (Query), keyword (Key) and value (Value), respectively.

[0146] According to an embodiment of the present disclosure, three three-dimensional tensors are input into a multi-headed self-attention mechanism layer (MSA), and a sixth matrix is ​​output. The calculation process of formulas (5) to (7) is used in the multi-headed self-attention mechanism layer.

[0147] Multi_head(Q,K,V)=concat(head1,...,head 12 )W o (5)

[0148]

[0149]

[0150] Among them, W o is a trainable weight matrix.

[0151] According to an embodiment of the present disclosure, a seventh matrix is ​​generated according to the sixth matrix and the third matrix. The seventh matrix is ​​input into the second normalization layer, and an eighth matrix is ​​output. The eighth matrix is ​​input into a multilayer perceptron, and a ninth matrix is ​​output. According to an embodiment of the present disclosure, a multilayer perceptron (MLP) may include two linear layers, a Gaussian error linear unit (GELU), and two Dropout units. GELU is used as an activation function, and the Dropout unit is used to prevent the sound recognition model from overfitting.

[0152] The model encoder may include a Transformer Encoder layer, whose output and input feature shapes are not transformed. The fourth matrix output by the model encoder is input into the multi-layer perceptron head MLP head. The perceptron head includes two linear layers and a Tanh activation function. It first extracts the class token∈(1,768) vector from the features processed by the model encoder and uses it as the input of the linear layer. The second linear layer outputs the possible probabilities of multiple categories, and finally outputs the category with the maximum probability of classification as the prediction result.

[0153] According to an embodiment of the present disclosure, the logarithmic Mel frequency spectrum feature includes a time dimension and a frequency dimension.

[0154] According to an embodiment of the present disclosure, processing the logarithmic Mel spectrum features according to a preset signal processing rule to obtain an initial two-dimensional matrix may include the following operations:

[0155] The frequency dimension is processed according to the preset frequency condition to obtain the target frequency dimension.

[0156] The time dimension is processed according to the preset time length condition to obtain the target time dimension.

[0157] Construct an initial two-dimensional matrix based on the target frequency dimension and the target time dimension.

[0158] According to an embodiment of the present disclosure, when the Vision Transformer model is used in the classifier recognition module, considering the characteristics of the Vision Transformer, the frequency dimension of the logarithmic Mel spectrum feature can be set to 224, thereby obtaining the target frequency dimension.

[0159] According to an embodiment of the present disclosure, in order to solve the problem of variable length of audio information, the audio information can be processed by using the preset time length conditions of padding short with zeros, cutting off long with zeros, and setting the number of time frames to 224, so as to obtain the target time dimension.

[0160] According to an embodiment of the present disclosure, an initial two-dimensional matrix of size 224 x 224 can be constructed according to the target frequency dimension and the target time dimension obtained above. According to an embodiment of the present disclosure, when training the classifier recognition module, a preset number of training audio information of infants aged between 1 and 6 months can be recorded, and preprocessed and feature extracted, and the extracted logarithmic Mel spectrum features are processed using preset signal processing rules to obtain an initial training two-dimensional matrix, thereby constructing a three-dimensional training tensor to facilitate training of the classifier recognition module.

[0161] According to an embodiment of the present disclosure, after the classifier recognition module is trained, the CRIED data set and self-recorded verification audio information of infants or special populations can be used to verify the accuracy of its prediction. Among them, the audio sampling rate in the CRIED data set is 32kHz, and the audio duration is variable from 0.4 seconds to 41 seconds. The sampling rate of the self-recorded audio is 44.1kHz, and the audio duration is variable from 23 seconds to 195 seconds. It is also preprocessed and feature extracted, and the extracted logarithmic Mel spectrum features are processed using preset signal processing rules to obtain an initial verification two-dimensional matrix, thereby constructing a three-dimensional verification tensor to facilitate verification of the classifier recognition module. Among them, the three-dimensional verification tensor can be (X, Δ T X, ), (X, Δ T X, ), (X, Δ F X, ), (X, Δ F X, ) and (X, ΔTX, Δ F The verification results are shown in Table 1.

[0162] Table 1

[0163]

[0164] As shown in Table 1, serial numbers 1 to 6 are models used in related technologies, serial numbers 7 to 11 are classifier recognition modules using the VisionTransformer model in the present disclosure, and serial number 12 is a technical solution combining the classifier recognition module of the present disclosure with the model parameters in the related technology, wherein UAR is the unweighted average recall rate and Acc is the accuracy rate.

[0165] From Table 1, we can see that based on (X, Δ T X, Δ F The recognition system that combines the feature X) and the ViT model surpasses the classification performance of other systems. Moreover, when the pre-trained model is loaded for fine-tuning, the UAR index of the model No. 12 is improved by 2.7% and the Acc index is improved by 2.6% compared with the model No. 11. T X, ) and (X, Δ F X, ) two features, it can be seen that the difference of time dimension and frequency dimension has similar performance. T X, )(or(X, Δ F X, )) and (X, Δ T X, )(or(X, Δ F X, )) features, it can be seen that based on making a difference in the time dimension (or frequency dimension), making a second difference in the same dimension has better performance than making a second difference in another dimension.

[0166] Figure 5 The confusion matrix diagram of the UAR indicator according to an embodiment of the present disclosure is schematically shown.

[0167] like Figure 5 As shown in the figure, Neutral represents calmness, Crying represents crying, and Fussing represents irritability. The horizontal axis represents the real state of infants or special populations, and the vertical axis represents the predicted state of infants or special populations.

[0168] Figure 5The confusion matrix in shows the classification effects of different categories. Since the “Neutral” category has more samples and the “Crying” and “Fussing” categories have more blurred classification boundaries, the classification performance of the latter two categories is somewhat different from that of the first category in terms of the UAR index. The present disclosure has made significant improvements to this classification imbalance phenomenon.

[0169] According to the embodiments of the present disclosure, the prediction accuracy of the sound recognition method of the present disclosure is verified by using self-recorded verification audio information of infants or special groups, and the results shown in Table 2 are obtained.

[0170] Table 2

[0171]

[0172] Regarding the judgment of infants' needs, the classification results in Table 2 are not very accurate due to the small number of self-recorded data samples, serious data category imbalance, and sample labels are marked by infants' relatives (leading to unreliability of sample labels). However, the results in Table 2 still show the effectiveness of the method disclosed in this disclosure, which is better than other models and features.

[0173] Figure 6 The block diagram schematically shows a voice recognition device according to an embodiment of the present disclosure.

[0174] like Figure 6 As shown, the sound recognition device 600 may include an acquisition module 610 , a pre-processing module 620 , a feature extraction module 630 , a first processing module 640 , a second processing module 650 , a generation module 660 and a prediction module 670 .

[0175] The acquisition module 610 is used to acquire audio information of user objects whose ages are within the target age range, wherein the user objects express their needs through voice characteristics.

[0176] The preprocessing module 620 is used to preprocess the audio information to obtain target audio information.

[0177] The feature extraction module 630 is used to extract features from the target audio information to obtain logarithmic Mel spectrum features.

[0178] The first processing module 640 is used to process the logarithmic Mel spectrum features according to a preset signal processing rule to obtain an initial two-dimensional matrix.

[0179] The second processing module 650 is used to process the initial two-dimensional matrix using a preset difference processing rule to obtain a target two-dimensional matrix.

[0180] The generation module 660 is used to generate a three-dimensional tensor according to the target two-dimensional matrix and the initial two-dimensional matrix.

[0181] The prediction module 670 is used to input the three-dimensional matrix into the classifier recognition module and output a prediction result, wherein the prediction result is used to characterize the needs of the user object.

[0182] According to the embodiments of the present disclosure, the above-mentioned logarithmic Mel-spectrum frequency features are processed by using preset signal processing rules to obtain an initial two-dimensional matrix, and then the obtained initial two-dimensional matrix is ​​processed by using preset differential processing rules, and the two-dimensional matrix obtained after the differential processing is combined with the initial two-dimensional matrix to generate a three-dimensional tensor, and the three-dimensional tensor is input into a classifier recognition module to predict user needs. This technical means at least partially overcomes the technical problem that it is difficult to judge user needs through the voice characteristics of users in the target age group, thereby improving the accuracy of identifying user needs through voice characteristics.

[0183] According to an embodiment of the present disclosure, the initial two-dimensional matrix includes a target frequency dimension and a target time dimension.

[0184] According to an embodiment of the present disclosure, the second processing module 650 may include a differential processing unit.

[0185] The differential processing unit is used to perform differential processing on the initial two-dimensional matrix in the target frequency dimension and the target time dimension to obtain the target two-dimensional matrix

[0186] According to an embodiment of the present disclosure, the pre-processing module 620 may include a down-sampling unit, a noise reduction unit, and a silence suppression unit.

[0187] The downsampling unit is used to downsample the sampling rate of the audio information to obtain the first audio information.

[0188] The noise reduction unit is used to perform noise reduction processing on the first audio information to obtain second audio information.

[0189] The silence suppression unit is used to perform silence suppression processing on the second audio information to obtain target audio information.

[0190] According to an embodiment of the present disclosure, the silence suppression unit may include a silence suppression subunit.

[0191] The silence suppression subunit is used to process the silence segment in the second audio information using the voice activity detection method to obtain the target audio information.

[0192] According to an embodiment of the present disclosure, the classifier recognition module may include a sound recognition model, which includes a feature embedding layer, a model encoder and a multi-layer perceptron head.

[0193] According to an embodiment of the present disclosure, the prediction module 670 may include a cutting unit, a first input unit, a splicing unit, a obtaining unit, a second input unit, and a third input unit.

[0194] The cutting unit is used to cut the three-dimensional tensor according to a preset cutting rule to obtain multiple sub-tensors.

[0195] The first input unit is used to input multiple sub-tensors into the feature embedding layer and output a first matrix.

[0196] The concatenation unit is used for concatenating the preset vector on the first matrix to obtain the second matrix.

[0197] The obtaining unit is used to obtain a third matrix according to the second matrix and a preset position coding matrix.

[0198] The second input unit is used to input the third matrix into the model encoder and output the fourth matrix.

[0199] The third input unit is used to input the fourth matrix into the multi-layer perceptron head and output the prediction result.

[0200] According to an embodiment of the present disclosure, the second input unit may include a first input subunit, an iteration subunit, and a determination subunit.

[0201] The first input subunit is used to input the third matrix into the model encoder and output a new third matrix.

[0202] The iteration subunit is used for iteratively using the new third matrix as the third matrix when the preset condition is not met, so as to input the third matrix into the model encoder.

[0203] The determining subunit is used to determine the new third matrix as the fourth matrix when a preset condition is met.

[0204] According to an embodiment of the present disclosure, the model encoder may include a first normalization layer, a multi-head attention mechanism layer, a second normalization layer and a multi-layer perceptron.

[0205] According to an embodiment of the present disclosure, the second input unit may include a second input subunit, a processing subunit, an obtaining subunit, a third input subunit, a first generating subunit, a fourth input subunit, a fifth input subunit and a second generating subunit.

[0206] The second input subunit is used to input the third matrix into the first normalization layer to obtain the normalized third matrix.

[0207] The processing subunit is used to process the normalized third matrix using a linear layer to obtain a target feature matrix.

[0208] The subunit is used to obtain three three-dimensional tensors according to the target feature matrix.

[0209] The third input sub-unit is used to input three three-dimensional tensors into the multi-head attention mechanism layer and output the fifth matrix.

[0210] The first generating subunit is used to generate a sixth matrix according to the fifth matrix and the third matrix.

[0211] The fourth input subunit is used to input the sixth matrix into the second normalization layer and output the seventh matrix.

[0212] The fifth input subunit is used to input the seventh matrix into the multilayer perceptron and output the eighth matrix.

[0213] The second generating subunit is used to generate a fourth matrix according to the eighth matrix and the sixth matrix.

[0214] According to an embodiment of the present disclosure, the logarithmic Mel frequency spectrum feature may include a time dimension and a frequency dimension.

[0215] According to an embodiment of the present disclosure, the first processing module 640 may include a first processing unit, a second processing unit, and a third processing unit.

[0216] The first processing unit is used to process the frequency dimension according to a preset frequency condition to obtain a target frequency dimension.

[0217] The second processing unit is used to process the time dimension according to a preset time condition to obtain a target time dimension.

[0218] The third processing unit is used to construct an initial two-dimensional matrix according to the target frequency dimension and the target time dimension.

[0219] According to the modules, units, and sub-units of the embodiments of the present disclosure, any multiple or at least part of the functions of any multiple thereof can be implemented in one module. According to the modules, units, and sub-units of the embodiments of the present disclosure, any one or more can be split into multiple modules for implementation. According to the modules, units, and sub-units of the embodiments of the present disclosure, any one or more can be at least partially implemented as a hardware circuit, such as a field programmable gate array (Field Programmable Gate Array, FPGA), a programmable logic array (Programmable Logic Arrays, PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), or can be implemented by hardware or firmware of any other reasonable way of integrating or encapsulating the circuit, or implemented in any one of the three implementation modes of software, hardware, and firmware or in any appropriate combination of any of them. Alternatively, according to the modules, units, and sub-units of the embodiments of the present disclosure, one or more can be at least partially implemented as a computer program module, and when the computer program module is run, the corresponding function can be performed.

[0220] For example, any multiple of the acquisition module 610, the preprocessing module 620, the feature extraction module 630, the first processing module 640, the second processing module 650, the generation module 660, and the prediction module 670 can be combined in one module / unit / sub-unit for implementation, or any one of the modules / units / sub-units can be split into multiple modules / units / sub-units. Alternatively, at least part of the functions of one or more of these modules / units / sub-units can be combined with at least part of the functions of other modules / units / sub-units and implemented in one module / unit / sub-unit. According to an embodiment of the present disclosure, at least one of the acquisition module 610, the preprocessing module 620, the feature extraction module 630, the first processing module 640, the second processing module 650, the generation module 660 and the prediction module 670 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application-specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or by any one of the three implementation methods of software, hardware and firmware or by an appropriate combination of any of them. Alternatively, at least one of the acquisition module 610, the preprocessing module 620, the feature extraction module 630, the first processing module 640, the second processing module 650, the generation module 660 and the prediction module 670 can be at least partially implemented as a computer program module, and when the computer program module is run, the corresponding function can be performed.

[0221] It should be noted that the voice recognition device part in the embodiment of the present disclosure corresponds to the voice recognition method part in the embodiment of the present disclosure. The description of the voice recognition device part specifically refers to the voice recognition method part, which will not be repeated here.

[0222] Figure 7 A block diagram of an electronic device suitable for implementing the method described above according to an embodiment of the present disclosure is schematically shown. Figure 7 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0223] like Figure 7As shown, the electronic device 700 according to the embodiment of the present disclosure includes a processor 701, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 702 or the program loaded from the storage part 708 to the random access memory (RAM) 703. The processor 701 may include, for example, a general-purpose microprocessor (such as a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (for example, an application-specific integrated circuit (ASIC)), etc. The processor 701 may also include an onboard memory for caching purposes. The processor 701 may include a single processing unit or multiple processing units for performing different actions of the method flow according to the embodiment of the present disclosure.

[0224] In RAM 703, various programs and data required for the operation of electronic device 700 are stored. Processor 701, ROM 702 and RAM 703 are connected to each other via bus 704. Processor 701 performs various operations of the method flow according to the embodiment of the present disclosure by executing the program in ROM 702 and / or RAM 703. It should be noted that the program can also be stored in one or more memories other than ROM 702 and RAM 703. Processor 701 can also perform various operations of the method flow according to the embodiment of the present disclosure by executing the program stored in the one or more memories.

[0225] According to an embodiment of the present disclosure, the electronic device 700 may further include an input / output (I / O) interface 705, which is also connected to the bus 704. The system 700 may further include one or more of the following components connected to the I / O interface 705: an input section 706 including a keyboard, a mouse, etc.; an output section 707 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 708 including a hard disk, etc.; and a communication section 709 including a network interface card such as a LAN card, a modem, etc. The communication section 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the I / O interface 705 as needed. A removable medium 711, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 710 as needed, so that a computer program read therefrom is installed into the storage section 708 as needed.

[0226] According to an embodiment of the present disclosure, the method flow according to an embodiment of the present disclosure can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program contains a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 709, and / or installed from the removable medium 711. When the computer program is executed by the processor 701, the above-mentioned functions defined in the system of the embodiment of the present disclosure are executed. According to an embodiment of the present disclosure, the system, equipment, device, module, unit, etc. described above can be implemented by a computer program module.

[0227] The present disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present disclosure is implemented.

[0228] According to an embodiment of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium. For example, it may include, but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM (Erasable Programmable Read Only Memory, EPROM) or flash memory), a portable compact disk read-only memory (Computer Disc Read-Only Memory, CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus or a device.

[0229] For example, according to an embodiment of the present disclosure, the computer-readable storage medium may include the ROM 702 and / or the RAM 703 described above and / or one or more memories other than the ROM 702 and the RAM 703 .

[0230] The embodiments of the present disclosure also include a computer program product, which includes a computer program, and the computer program contains program code for executing the method provided by the embodiments of the present disclosure. When the computer program product runs on an electronic device, the program code is used to enable the electronic device to implement the sound recognition method provided by the embodiments of the present disclosure.

[0231] When the computer program is executed by the processor 701, the above functions defined in the system / device of the embodiment of the present disclosure are executed. According to the embodiment of the present disclosure, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0232] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, and downloaded and installed through the communication part 709, and / or installed from the removable medium 711. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0233] According to an embodiment of the present disclosure, the program code for executing the computer program provided by the embodiment of the present disclosure can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect through the Internet).

[0234] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram may represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box may also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions. It can be understood by those skilled in the art that the features recorded in the various embodiments and / or claims of the present disclosure can be combined and / or combined in a variety of ways, even if such a combination or combination is not explicitly recorded in the present disclosure. In particular, without departing from the spirit and teaching of the present disclosure, the features described in the various embodiments and / or claims of the present disclosure may be combined and / or combined in a variety of ways. All of these combinations and / or combinations fall within the scope of the present disclosure.

[0235] The embodiments of the present disclosure are described above. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of the present disclosure. Although the embodiments are described above separately, this does not mean that the measures in the various embodiments cannot be used in combination to advantage. The scope of the present disclosure is defined by the attached claims and their equivalents. Without departing from the scope of the present disclosure, those skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present disclosure.

Claims

1. A sound recognition method, comprising: Acquiring audio information of a user object whose age is within a target age group, wherein the user object expresses a demand through voice characteristics; Preprocessing the audio information to obtain target audio information; Extracting features of the target audio information to obtain logarithmic Mel frequency spectrum features; Processing the logarithmic Mel spectrum feature according to a preset signal processing rule to obtain an initial two-dimensional matrix, wherein the initial two-dimensional matrix includes a target frequency dimension and a target time dimension; Performing difference processing on the initial two-dimensional matrix in the target frequency dimension and the target time dimension respectively to obtain a target two-dimensional matrix; Generate a three-dimensional tensor according to the target two-dimensional matrix and the initial two-dimensional matrix; and The three-dimensional tensor is input into a classifier recognition module, and a prediction result is output, wherein the prediction result is used to characterize the needs of the user object.

2. The method according to claim 1, wherein: The preprocessing of the audio information to obtain target audio information includes: Downsampling the sampling rate of the audio information to obtain first audio information; Performing noise reduction processing on the first audio information to obtain second audio information; Perform silence suppression processing on the second audio information to obtain the target audio information.

3. The method according to claim 2, wherein: The performing silence suppression processing on the second audio information to obtain the target audio information includes: The silent segment in the second audio information is processed by using a voice activity detection method to obtain the target audio information.

4. The method according to claim 1, wherein the classifier recognition module comprises a sound recognition model, wherein the sound recognition model comprises a feature embedding layer, a model encoder and a multi-layer perceptron head; in, The step of inputting the three-dimensional tensor into a classifier recognition module and outputting a prediction result comprises: The three-dimensional tensor is cut according to the preset cutting rules to obtain multiple sub-tensors Input the plurality of sub-tensors into the feature embedding layer, and output a first matrix; splicing the preset vectors on the first matrix to obtain a second matrix; Obtaining a third matrix according to the second matrix and a preset position coding matrix; Input the third matrix into the model encoder and output a fourth matrix; The fourth matrix is ​​input into the multi-layer perceptron head, and the prediction result is output.

5. The method according to claim 4, wherein: The step of inputting the third matrix into the model encoder and outputting a fourth matrix comprises: Input the third matrix into the model encoder and output a new third matrix; If the preset condition is not satisfied, iteratively use the new third matrix as the third matrix to input the third matrix into the model encoder; When a preset condition is met, the new third matrix is ​​determined as the fourth matrix.

6. The method according to claim 4, wherein the model encoder comprises a first normalization layer, a multi-head attention mechanism layer, a second normalization layer and a multi-layer perceptron; in, The step of inputting the third matrix into the model encoder and outputting a fourth matrix comprises: Inputting the third matrix into the first normalization layer to obtain a normalized third matrix; Processing the normalized third matrix using a linear layer to obtain a target feature matrix; According to the target feature matrix, three three-dimensional tensors are obtained; Input the three three-dimensional tensors into the multi-head attention mechanism layer, and output a fifth matrix; Generate a sixth matrix according to the fifth matrix and the third matrix; Input the sixth matrix into the second normalization layer, and output a seventh matrix; Input the seventh matrix into the multilayer perceptron and output an eighth matrix; The fourth matrix is ​​generated according to the eighth matrix and the sixth matrix.

7. The method according to claim 1, wherein the logarithmic Mel spectrum feature comprises a time dimension and a frequency dimension; in, The process of processing the logarithmic Mel spectrum feature according to a preset signal processing rule to obtain an initial two-dimensional matrix includes: Processing the frequency dimension according to a preset frequency condition to obtain a target frequency dimension; Processing the time dimension according to a preset time condition to obtain a target time dimension; The initial two-dimensional matrix is ​​constructed according to the target frequency dimension and the target time dimension.

8. A sound recognition device, comprising: An acquisition module, used to acquire audio information of a user object whose age is within a target age range, wherein the user object expresses the demand through voice characteristics; A preprocessing module, used for preprocessing the audio information to obtain target audio information; A feature extraction module, used to extract features from the target audio information to obtain logarithmic Mel spectrum features; A first processing module, used for processing the logarithmic Mel spectrum feature according to a preset signal processing rule to obtain an initial two-dimensional matrix; The second processing module is used to process the initial two-dimensional matrix using a preset difference processing rule to obtain a target two-dimensional matrix, and the second processing module includes: A differential processing unit, used for performing differential processing on the initial two-dimensional matrix in a target frequency dimension and a target time dimension respectively, to obtain a target two-dimensional matrix; A generating module, configured to generate a three-dimensional tensor according to the target two-dimensional matrix and the initial two-dimensional matrix; and The prediction module is used to input the three-dimensional tensor into a classifier recognition module and output a prediction result, wherein the prediction result is used to characterize the needs of the user object.

9. An electronic device, comprising: one or more processors; a memory for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 7.

Images