Crying sound detection network training method and device, electronic equipment and storage medium

Abstract

The present application is suitable for the field of computer technology, and provides a crying sound detection network training method and device, electronic equipment and storage medium, the method comprises the following steps: labeling and augmenting the audio data obtained including the baby crying sound to obtain a training data set; inputting the training data set into a pre-constructed model training network including an encoder, a decoder and an auxiliary training module, training the crying sound detection network composed of the encoder and the decoder to obtain a trained crying sound detection network; wherein the encoder, the decoder and the auxiliary training module are all convolutional neural networks, so that the auxiliary training module is added during the training of the crying sound detection network, the feature extraction capability of the encoder during the crying sound detection is improved, important information is fully retained for the subsequent decoder, and the auxiliary training module does not participate in the detection process during the detection, thereby improving the baby crying sound detection effect of the crying sound detection network without increasing the parameter amount.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of computers, and particularly relates to a crying sound detection network training method and device, electronic equipment and a storage medium. BACKGROUND

[0002] In order to solve the parenting problems of new generation parents, infant auxiliary equipment intelligent IPC (IP Camera, network camera) based on AI technology has emerged as the times require, especially for newborn babies, crying is the only way of communication with the outside world, in order to better monitor the state of the baby, infant crying sound detection network training has become a standard of contemporary intelligent IPC.

[0003] However, the existing crying sound detection algorithm has low accuracy, and the detection duration is relatively long, generally lasting 30s-40s to be detected, and the application environment requirement is relatively high, most products can only achieve ideal detection effect in a quiet indoor environment, and the detection performance sharply decreases when the environment sound is slightly complex. SUMMARY

[0004] The present application aims to provide a crying sound detection network training method, device, electronic equipment and storage medium, which aims to solve the problem that the infant sound detection model trained based on a small amount of data set in the prior art has insufficient detection performance.

[0005] In one aspect, the present application provides a crying sound detection network training method, which comprises the following steps:

[0006] annotating and augmenting the obtained audio data including infant crying sound to obtain a training data set;

[0007] inputting the training data set into a pre-constructed model training network including an encoder, a decoder and an auxiliary training module, training a crying sound detection network composed of the encoder and the decoder, and obtaining a trained crying sound detection network;

[0008] The encoder, the decoder and the auxiliary training module are all convolutional neural networks, and the inputs of the decoder and the auxiliary training module are the training data set processed by the encoder.

[0009] Optionally, the step of annotating and augmenting the obtained audio data including infant crying sound to obtain a training data set comprises:

[0010] annotating the audio data using weak labels to obtain weak labels of audio levels corresponding to the audio data;

[0011] performing first augmentation processing on the audio data corresponding to each weak label;

[0012] Perform a second augmentation process on the audio data after the first augmentation process, and set the audio data after the second augmentation process as the training data set.

[0013] Optionally, the step of inputting the training data set into a pre-built model training network including an encoder, a decoder, and an auxiliary training module, and training the crying detection network composed of the encoder and the decoder to obtain the trained crying detection network includes:

[0014] The encoder and the auxiliary training module are trained using a mean square error loss function. When the mean square error loss function converges, the encoder, the decoder and the auxiliary training module are trained using the mean square error loss function and the cross entropy loss function to obtain the trained crying detection network.

[0015] Optionally, the encoder includes a plurality of two-dimensional convolution modules and two-dimensional pooling modules connected in sequence, wherein the two-dimensional convolution module includes a batch normalization layer, a two-dimensional convolution layer and an activation function, and the two-dimensional pooling module includes a two-dimensional pooling layer. The input of the encoder is a training data set obtained after the labeling and augmentation processing.

[0016] Optionally, the decoder includes a long short-term memory layer, a fully connected layer, an activation layer and a temporal pooling layer connected in sequence, the long short-term memory layer is used to extract the correlation information between the time domain features of the audio in the training data set received by the decoder, the fully connected layer is used to classify the extracted correlation information at the time frame level, and use the activation layer to perform activation processing to obtain the probability value of the time frame level corresponding to the correlation information, and the temporal pooling layer is used to convert the probability value of the time frame level into a probability value at the audio level.

[0017] Optionally, the auxiliary training module includes a plurality of deconvolution modules and two-dimensional convolution modules connected in sequence, wherein the deconvolution module includes a deconvolution layer, and the two-dimensional convolution module includes a batch normalization layer, a two-dimensional convolution layer and an activation function. The auxiliary training module is used to perform smoothing filtering on the training data set processed by the encoder so that the decoder learns the characteristics of audio classification.

[0018] In another aspect, the present invention provides a crying sound detection method, comprising the following steps:

[0019] The obtained audio data including the baby's crying is detected by a trained crying detection network to obtain a detection result of the baby's crying, and the crying detection network is trained by the aforementioned crying detection network training method.

[0020] In another aspect, the present application provides a cry detection network training device, the device comprising:

[0021] a data set acquisition unit configured to label and augment the acquired audio data comprising infant cries to obtain a training data set;

[0022] a network training unit configured to input the training data set into a pre-constructed model training network comprising an encoder, a decoder and an auxiliary training module, train a cry detection network composed of the encoder and the decoder, and obtain a trained cry detection network;

[0023] wherein the encoder, the decoder and the auxiliary training module are all convolutional neural networks, and the input of the decoder and the auxiliary training module is the training data set processed by the encoder.

[0024] In another aspect, the present application further provides an electronic device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program.

[0025] In another aspect, the present application further provides a computer readable storage medium storing a computer program, wherein the computer program is executable by a processor to implement the steps of the above method.

[0026] The present application labels and augments the acquired audio data comprising infant cries to obtain a training data set, inputs the training data set into a pre-constructed model training network comprising an encoder, a decoder and an auxiliary training module, trains a cry detection network composed of the encoder and the decoder, and obtains a trained cry detection network, wherein the encoder, the decoder and the auxiliary training module are all convolutional neural networks, and the input of the decoder and the auxiliary training module is the training data set processed by the encoder, thereby adding the auxiliary training module during the training of the cry detection network, improving the feature extraction capability of the encoder during the cry detection, fully retaining important information for the subsequent decoder, and improving the infant cry detection effect of the cry detection network without increasing the parameter amount. BRIEF DESCRIPTION OF DRAWINGS

[0027] Figure 1A is the implementation flowchart of the cry detection network training method provided by the first embodiment of the present application;

[0028] Figure 1B shows the feature maps output by the encoder and the auxiliary training module respectively according to the first embodiment of the present application;

[0029] Figure 1C Schematic diagram of the structure of the model training network provided in the first embodiment of the present invention;

[0030] Figure 1D 1 is a schematic diagram of the structure of a crying detection network provided in Example 1 of the present invention;

[0031] Figure 2 Flowchart of the implementation of the crying detection method provided in the second embodiment of the present invention;

[0032] Figure 3 2 is a schematic diagram of the structure of a crying sound detection network training device provided in Embodiment 3 of the present invention;

[0033] Figure 4 is a structural diagram of a crying sound detection device provided in a fourth embodiment of the present invention; and

[0034] Figure 5 It is a structural diagram of an electronic device provided in Example 5 of the present invention. DETAILED DESCRIPTION

[0035] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0036] The following describes the specific implementation of the present invention in detail with reference to specific embodiments:

[0037] Example One:

[0038] FIG1 shows the implementation process of the crying detection network training method provided in the first embodiment of the present invention. For ease of explanation, only the part related to the embodiment of the present invention is shown, which is described in detail as follows:

[0039] In step S101 , the acquired audio data including the baby crying sound is labeled and augmented to obtain a training data set.

[0040] Embodiments of the present application are applicable to electronic devices, which can be terminal devices such as an intelligent IPC (IP Camera, network camera) for infant auxiliary devices, a mobile phone, a tablet computer, a wearable device, a vehicle-mounted device, a monitoring camera, an augmented reality (AR) / virtual reality (VR) device, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), and the like. Embodiments of the present application do not limit the specific type of electronic devices.

[0041] In the embodiments of the present application, when the audio data including the baby crying sound is labeled, the audio data can be classified and labeled according to whether the baby crying sound is contained in the audio data. For ease of description, the labels of the labeling can be denoted as a first label and a second label, so as to obtain the classification labels of the audio data, that is, the audio data is divided into audio data with the first label and audio data with the second label. The audio data with the first label indicates that the audio data contains the baby crying sound, and the audio data with the second label indicates that the audio data does not contain the baby crying sound. Further, for ease of description, the audio data corresponding to the first label is referred to as baby crying sound audio, and the audio data corresponding to the second label is referred to as non-baby crying sound audio. When the labeled audio data is augmented, the audio data can be speed-adjusted, noise-added, pitch-adjusted, and volume-adjusted, and the like, so as to generate training data sets similar to the audio data labeled with the crying sound but different from the audio data labeled with the crying sound. Then, the augmented audio data is used as the training data set, so as to expand the scale of the training data set by labeling and augmenting the original audio data, and improve the generalization ability of the crying sound detection network during subsequent training of the crying sound detection network.

[0042] In an embodiment, before the obtained audio data is labeled, the obtained audio data can be processed into single-channel wav format with a sampling rate of 16 kHz and a sampling precision of 16 bits, so as to save data storage space while obtaining audio data without distortion. When the processed audio data is labeled, preferably, the audio data is labeled by using a weak label to obtain the weak label of the audio level corresponding to the audio data, so as to more fully and effectively utilize the input audio data while reducing the labeling workload.

[0043] In an embodiment, preferably, the first augmentation processing is speed adjustment, and the second augmentation processing is noise addition, pitch adjustment, and volume adjustment, so that the audio is augmented in a more flexible manner while ensuring uniform audio length. Specifically, the corresponding infant crying audio under the first label is subjected to speed adjustment. When the length of the audio after speed adjustment is less than the fixed length, zero padding operation can be performed, and when the length of the audio after speed adjustment is greater than the fixed length, truncation operation can be performed. The infant crying audio after the first augmentation processing is subjected to noise addition, pitch adjustment, and volume adjustment. The order of noise addition, pitch adjustment, and volume adjustment, and the corresponding augmentation proportion coefficient and augmentation parameter are determined according to the actual situation of the task. Thus, the corresponding infant crying audio under the first label is augmented by using different augmentation manners, so that the complexity of the audio data for training is increased, and the generalization ability of the crying detection network is improved. It should be noted that the corresponding non-infant crying audio under the second label is relatively easy to collect and has a large quantity, so that only the infant crying audio needs to be augmented, and the non-infant crying audio does not need to be augmented. When the proportion of the augmented infant crying audio and the non-infant crying audio is not less than a preset proportion, the augmentation of the infant crying audio is stopped, so that the complexity of the infant crying audio data is moderately increased, the computational complexity and storage space are not excessively increased, and the training effect of the crying detection network is ensured.

[0044] In step S102, the training data set is input into a model training network including an encoder, a decoder, and an auxiliary training module, the crying detection network composed of the encoder and the decoder is trained, and a trained crying detection network is obtained.

[0045] In the embodiment of the present application, the model training network is used for training the cry detection network composed of an encoder and a decoder, and the model training network is composed of the encoder, the decoder and an auxiliary training module in the cry detection network, wherein the encoder, the decoder and the auxiliary training module are all convolutional neural networks, the encoder is used for feature extraction of different dimensions on the input training data set, the decoder is used for time frame level classification on the output of the encoder to obtain a time frame level probability value, and then the time frame level probability value is converted into an audio level probability output, and the auxiliary training module is used for smoothing filtering on the output of the encoder to assist the decoder to learn the main features of the time frame level classification. When the training data set obtained through the step S101 is input into the model training network for training, first, the encoder and the auxiliary training module are trained by using the mean square error loss function, the auxiliary training module helps the encoder to better learn the key information in the audio, so that the encoder learns better. Then, when the mean square error loss function converges, the encoder, the decoder and the auxiliary training module are trained at the same time by using the mean square error loss function and the cross-entropy loss function, and the trained cry detection network is obtained, so that the cry detection network converges at a faster speed.

[0046] In a preferred embodiment, the model training network comprises an encoder, a decoder and an auxiliary training module, the decoder and the auxiliary training module are connected in parallel on the encoder, and the output of the encoder is the input of the decoder and the auxiliary training module. The encoder is used to extract features of different dimensions from the input training data set, and the encoder comprises a plurality of two-dimensional convolution modules and two-dimensional pooling modules connected in sequence, the two-dimensional convolution module comprises a batch normalization layer, a two-dimensional convolution layer and an activation function, and the two-dimensional pooling module comprises a two-dimensional pooling layer. The decoder is used to classify the output of the encoder at the time frame level to obtain a time frame level probability value, and then convert the time frame level probability value into an audio level probability output, and the decoder comprises a long short-term memory layer, a full connection layer, an activation layer and a time sequence pooling layer connected in sequence, wherein the long short-term memory layer is used to extract the association information between the time domain features of the audio in the training data set received by the decoder, the full connection layer is used to classify the extracted association information at the time frame level, and the activation layer is used to activate the association information to obtain a time frame level probability value corresponding to the association information, and the time sequence pooling layer is used to convert the time frame level probability value into an audio level probability value. The auxiliary training module is used to perform smoothing filtering on the output of the encoder to assist the decoder to learn the features of the time frame level classification, and the auxiliary training module comprises a plurality of deconvolution modules and two-dimensional convolution modules connected in sequence, the deconvolution module comprises a deconvolution layer, and the two-dimensional convolution module has the same structure as the two-dimensional convolution module in the encoder, which will not be described here. In this way, when the training data set is input into the model training network for training, first, only the encoder and the auxiliary training module are trained by using the mean square error loss function, and the auxiliary training module enables the encoder to better learn the key information related to classification, and also plays a certain regularization role. After that, when the mean square error loss function converges, the encoder, the decoder and the auxiliary training module are trained simultaneously by using the mean square error loss function and the cross-entropy loss function, and the trained cry detection network is obtained, so that the cry detection network converges at a faster speed.

[0047] Specifically, when the encoder performs feature extraction, first, the input training data set is normalized by using a batch normalization layer, so as to improve the learning rate and the training speed of the encoder, and then a two-dimensional convolution is performed on the normalized training data set, so as to improve the feature difference between the infant crying audio and the non-infant crying audio in the training data set, and then an activation function is used to activate the convolutional training data set, so as to increase the non-linear relationship between the layers of the encoder, thereby completing the different tasks of each layer. When training the decoder, the long short-term memory layer is used to extract the correlation information between the audio time domains, the full connection layer is used to classify the correlation information at the time frame level, the activation layer is used to activate the time frame level classification, and the time frame level probability value is obtained, and finally the time frame level probability value is converted into an audio level probability by the time sequence pooling layer and then output. The time frame level is not the same concept as the audio level frame obtained after the audio is labeled. As an example, assuming that the audio feature when labeling the audio is 501*128, and after the full connection layer operation of the decoder, the output audio feature is 62*2, that is, the frame after labeling corresponds to 501, and the time frame obtained after the time frame level classification of the decoder is 62. When training the auxiliary training module, the audio frequency domain feature is deconvoluted by using the deconvolution layer, the time domain feature of the audio is obtained, and then the time domain feature of the audio is filtered to filter out the noise in the audio and obtain more audio detail features, so that the audio feature is smoother, and thus it is beneficial for the decoder to learn and classify the smooth audio feature. As an example, the effect of filtering the audio processed by the encoder by the auxiliary training module is as shown in the upper and lower graphs of FIGS. Figure 1B Figure 1B The upper graph of FIG. is the feature map after the encoder extracts the audio feature, and the lower graph is the feature map obtained after the auxiliary training module filters the audio after feature extraction, that is, the output of the auxiliary training module. It can be seen that the output of the auxiliary training module is smoother than the output of the encoder.

[0048] ​In a specific embodiment, when only the encoder and the auxiliary training module are trained using the mean square error loss function, the loss function of the model training network can be expressed as loss = loss1, where loss represents the loss function of the model training network, and loss1 represents the mean square error loss function in the auxiliary training module. When the mean square error loss function converges, the mean square error loss function and the cross entropy loss function are used to train the encoder, decoder and auxiliary training module of the model training network at the same time. At this time, the loss function of the model training network can be expressed as loss = loss1*0.001+loss2, where loss represents the loss function of the model training network, loss1 represents the mean square error loss function in the auxiliary training module, and loss2 represents the cross entropy loss function in the decoder. Through the above training method, the probability of the audio level output by the decoder can be made closer to the ratio of the baby crying audio corresponding to the first label to the non-baby crying audio corresponding to the second label, while accelerating the convergence of the decoder and the accuracy of the decoder in classifying audio time domain features, thereby helping the convergence of the model training network.

[0049] As an example, Figure 1C The structure of the model training network provided by the embodiment of the present invention is shown. Figure 1D Shown by Figure 1C The structure of the crying detection network obtained by training the model training network is shown in FIG. Figure 1C As shown, the model training network includes an encoder, a decoder and an auxiliary training module connected in sequence. The encoder includes multiple two-dimensional convolution modules (Block2D) and two-dimensional pooling modules (Pool2d) connected in sequence, wherein the two-dimensional convolution module (Block2D) includes a batch normalization layer (BN), a two-dimensional convolution layer (Conv2D) and an activation function (Relu), and the two-dimensional pooling module includes a two-dimensional pooling layer. The decoder includes a long short-term memory layer, a fully connected layer, an activation layer and a temporal pooling layer connected in sequence. The auxiliary training module includes multiple deconvolution modules and a two-dimensional convolution module (Block2D) connected in sequence. The deconvolution module includes a deconvolution layer. The structure of the two-dimensional convolution module is the same as that of the two-dimensional convolution module in the encoder, and will not be repeated here. Figure 1D As shown in the figure, the trained crying detection network consists of an encoder and a decoder, and the encoder and decoder are composed of Figure 1C The model training network shown is trained.

[0050] In the embodiment of the present application, the training data set is input into the model training network for training. In the training, the mean square error loss function is first used to train only the encoder and the auxiliary training module in the model training network. When the mean square error loss function converges, the mean square error loss function and the cross-entropy loss function are used to simultaneously train the encoder, the decoder and the auxiliary training module in the model training network, so as to obtain the trained cry detection network. Through the above training method, the training time of the cry detection network can be shortened, the convergence of the cry detection network can be accelerated, and thus the training effect of the cry detection network can be improved.

[0051] Example Two:

[0052] Figure 2 The implementation process of the cry detection method provided in the second embodiment of the present application is shown. For the convenience of description, only the parts related to the embodiments of the present application are shown, and the details are as follows:

[0053] In step S201, the audio data obtained and including the infant cry is labeled and augmented to obtain a training data set;

[0054] In step S202, the training data set is input into the pre-constructed model training network including the encoder, the decoder and the auxiliary training module, the cry detection network composed of the encoder and the decoder is trained, and the trained cry detection network is obtained.

[0055] In the embodiment of the present application, the specific implementation manners of steps S201 and S202 can refer to the corresponding descriptions of steps S101 and S102 in the first embodiment, and will not be described here.

[0056] In step S203, the audio data obtained and including the infant cry is detected by the trained cry detection network to obtain a detection result of the infant cry.

[0057] In the embodiment of the present application, the audio data obtained and including the infant cry is detected by the trained cry detection network to obtain a detection result of the infant cry. If the detection result contains the infant cry, the detection result and the detected audio data including the infant cry can be sent to a user client such as a mobile phone, a netbook or a vehicle-mounted device, so as to timely remind the guardian of the infant.

[0058] Example Three:

[0059] Figure 3 The structure of the cry detection network training device provided in the third embodiment of the present application is shown. For the convenience of description, only the parts related to the embodiments of the present application are shown, and the details are as follows:

[0060] The dataset obtaining unit 31 is configured to label and augment the obtained audio data including infant cries to obtain a training dataset.

[0061] The network training unit 32 is configured to input the training dataset into a pre-constructed model training network including an encoder, a decoder and an auxiliary training module, train a cry detection network composed of the encoder and the decoder to obtain a trained cry detection network, wherein the encoder, the decoder and the auxiliary training module are all convolutional neural networks, the encoder is configured to extract features of different dimensions from the input training dataset, and the inputs of the decoder and the auxiliary training module are the training dataset processed by the encoder.

[0062] In the embodiment of the present application, each unit of the cry detection network training apparatus can be realized by a corresponding hardware or software unit, and each unit can be an independent software or hardware unit, or can be integrated into a software or hardware unit, which does not limit the present application. The specific implementation of each unit of the cry detection network training apparatus can refer to the description of the foregoing method embodiment, which will not be repeated here.

[0063] Example Four:

[0064] Figure 4 The structure of the cry detection apparatus provided in the fourth embodiment of the present application is shown, only the parts related to the embodiments of the present application are shown for the convenience of description, which include:

[0065] The dataset obtaining unit 41 is configured to label and augment the obtained audio data including infant cries to obtain a training dataset.

[0066] The network training unit 42 is configured to input the training dataset into a pre-constructed model training network including an encoder, a decoder and an auxiliary training module, train a cry detection network composed of the encoder and the decoder to obtain a trained cry detection network, wherein the encoder, the decoder and the auxiliary training module are all convolutional neural networks, the encoder is configured to extract features of different dimensions from the input training dataset, and the inputs of the decoder and the auxiliary training module are the training dataset processed by the encoder.

[0067] The cry detection unit 43 is configured to detect the obtained audio data including infant cries by using the trained cry detection network to obtain a detection result of the infant cries.

[0068] In the embodiments of the present application, each unit of the cry detection apparatus can be implemented by a corresponding hardware or software unit, and each unit can be an independent software or hardware unit, or can be integrated into a software or hardware unit, which does not limit the present application. The specific implementation of each unit of the cry detection apparatus can refer to the description of the foregoing method embodiments, which will not be described here.

[0069] Example Five:

[0070] Figure 5 The structure of the electronic device provided by the fifth embodiment of the present application is shown, and only the parts related to the embodiments of the present application are shown for the convenience of description.

[0071] The electronic device 5 of the embodiments of the present application includes a processor 50, a memory 51, and a computer program 52 stored in the memory 51 and executable on the processor 50. The processor 50 implements the steps in the foregoing method embodiments when executing the computer program 52, for example Figure 1A the steps S101-S102 shown. Alternatively, the processor 50 implements the functions of each unit in the foregoing apparatus embodiments when executing the computer program 52, for example Figure 3 the functions of the units 31-32 shown.

[0072] In the embodiments of the present application, the training data set is input into the model training network for training. In the training, the mean square error loss function is first used to train only the encoder and the auxiliary training module in the model training network. When the mean square error loss function converges, the mean square error loss function and the cross-entropy loss function are used to train the encoder, the decoder, and the auxiliary training module in the model training network at the same time to obtain the trained cry detection network. The training time of the cry detection network can be shortened by the above training method, the convergence of the cry detection network can be accelerated, and thus the training effect of the cry detection network can be improved.

[0073] Example Six:

[0074] In the embodiments of the present application, a computer readable storage medium is provided, which stores a computer program. The computer program is executed by a processor to implement the steps in the foregoing method embodiments, for example, Figure 1A the steps S101-S102 shown. Alternatively, the computer program is executed by a processor to implement the functions of each unit in the foregoing apparatus embodiments, for example Figure 3 the functions of the units 31-32 shown.

[0075] In the embodiment of the present application, the training data set is input into the model training network for training. In the training, the mean square error loss function is first used to train only the encoder and the auxiliary training module in the model training network. When the mean square error loss function converges, the mean square error loss function and the cross-entropy loss function are used to simultaneously train the encoder, the decoder and the auxiliary training module in the model training network, to obtain the trained crying sound detection network. Through the above training manner, the training time of the crying sound detection network can be shortened, the convergence of the crying sound detection network can be accelerated, and thus the training effect of the crying sound detection network can be improved.

[0076] The computer readable storage medium of the embodiment of the present application can include any entity or device capable of carrying computer program code, recording medium, such as ROM / RAM, magnetic disk, optical disk, flash memory, etc.

[0077] The above only describes the preferred embodiments of the present application and is not intended to limit the present application. Any modification, equivalent replacement and improvement made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A crying detection network training method, characterized in that: The method comprises the following steps: The acquired audio data including baby crying is labeled and augmented to obtain a training data set; The training data set is input into a pre-constructed model training network comprising an encoder, a decoder and an auxiliary training module, a crying sound detection network composed of the encoder and the decoder is trained, and a trained crying sound detection network is obtained, the decoder and the auxiliary training module are connected in parallel on the encoder, the output of the encoder is the input of the decoder and the auxiliary training module, and the step of inputting the training data set into the pre-constructed model training network comprising the encoder, the decoder and the auxiliary training module, training the crying sound detection network composed of the encoder and the decoder, and obtaining the trained crying sound detection network comprises: training the encoder and the auxiliary training module by using a mean square error loss function, and when the mean square error loss function converges, training the encoder, the decoder and the auxiliary training module by using the mean square error loss function and a cross-entropy loss function. Wherein, when the encoder and the auxiliary training module are trained by using the mean square error loss function, the loss function of the model training network is the mean square error loss function in the auxiliary training module; when the mean square error loss function converges, the encoder, the decoder and the auxiliary training module are trained by using the mean square error loss function and the cross-entropy loss function, and the loss function of the model training network is represented as loss=loss1*0.001+loss2, wherein loss represents the loss function of the model training network, loss1 represents the mean square error loss function in the auxiliary training module, and loss2 represents the cross-entropy loss function in the decoder. The encoder, the decoder and the auxiliary training module are all convolutional neural networks, the input of the decoder and the auxiliary training module is the training data set processed by the encoder, and the auxiliary training module is used for smoothing filtering processing on the training data set processed by the encoder, so that the decoder learns the features of audio classification.

2. The method according to claim 1, wherein The step of labeling and augmenting the acquired audio data including baby crying to obtain a training data set comprises: The audio data is labeled by using weak labels to obtain weak labels corresponding to the audio levels of the audio data; The audio data corresponding to each weak label is subjected to first augmentation processing; The audio data after the first augmentation processing is subjected to second augmentation processing, and the audio data after the second augmentation processing is set as the training data set.

3. The method of claim 1, wherein, The encoder comprises a plurality of two-dimensional convolution modules and two-dimensional pooling modules connected in sequence, wherein the two-dimensional convolution module comprises a batch normalization layer, a two-dimensional convolution layer and an activation function, the two-dimensional pooling module comprises a two-dimensional pooling layer, and the input of the encoder is the training data set obtained after the labeling and augmentation processing.

4. The method of claim 1, wherein, The decoder comprises a long short-term memory layer, a full connection layer, an activation layer and a time sequence pooling layer connected in sequence, the long short-term memory layer is used to extract the correlation information between the time domain features of audio in a training data set received by the decoder, the full connection layer is used to classify the extracted correlation information at a time frame level, and the correlation information is activated by using the activation layer to obtain a probability value of the time frame level corresponding to the correlation information, and the time sequence pooling layer is used to convert the probability value of the time frame level into a probability value of an audio level.

5. The method of claim 1, wherein, The auxiliary training module comprises a plurality of anti-convolution modules and two-dimensional convolution modules connected in sequence, wherein the anti-convolution module comprises an anti-convolution layer, and the two-dimensional convolution module comprises a batch normalization layer, a two-dimensional convolution layer and an activation function.

6. A crying sound detection method characterized by, The method comprises the following steps: The audio data comprising the infant cry is detected by using the trained cry detection network to obtain the detection result of the infant cry, and the cry detection network is trained by using the cry detection network training method in any one of claims 1 to 5.

7. A crying sound detection network training apparatus, characterized by comprising: The device comprises: A data set acquisition unit is configured to label and augment the acquired audio data comprising the infant cry to obtain a training data set; A network training unit is configured to input the training data set into a pre-constructed model training network comprising an encoder, a decoder and an auxiliary training module, train a cry detection network composed of the encoder and the decoder, and obtain a trained cry detection network, wherein the decoder and the auxiliary training module are connected in parallel on the encoder, the output of the encoder is the input of the decoder and the auxiliary training module, and the training of the cry detection network composed of the encoder and the decoder by inputting the training data set into the pre-constructed model training network comprising the encoder, the decoder and the auxiliary training module comprises: training the encoder and the auxiliary training module by using a mean square error loss function, and training the encoder, the decoder and the auxiliary training module by using the mean square error loss function and a cross-entropy loss function when the mean square error loss function converges. In the training of the encoder and the auxiliary training module by using the mean square error loss function, the loss function of the model training network is the mean square error loss function in the auxiliary training module; when the mean square error loss function converges, the encoder, the decoder and the auxiliary training module are trained by using the mean square error loss function and the cross-entropy loss function, and the loss function of the model training network is represented as loss = loss1*0.001 + loss2, wherein loss represents the loss function of the model training network, loss1 represents the mean square error loss function in the auxiliary training module, and loss2 represents the cross-entropy loss function in the decoder. The encoder, the decoder and the auxiliary training module are all convolutional neural networks, the input of the decoder and the auxiliary training module is the training data set processed by the encoder, and the auxiliary training module is used for smoothing filtering processing on the training data set processed by the encoder, so that the decoder learns the features of audio classification.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor executes the computer program to realize the steps of the method in any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, the computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 8. The computer program is executed by the processor to realize the steps of the method in any one of claims 1 to 6.

Images