A baby crying detection method, apparatus and electronic device

Abstract

The application discloses a baby crying detection method and device and electronic equipment. The method comprises the following steps: calculating the spectrum, fundamental frequency and voice activity detection result of a new frame of voice signal each time a new frame of voice signal is acquired; updating a pre-established matrix set according to the spectrum, fundamental frequency and voice activity detection result of the new frame of voice signal according to the first-in first-out principle; calculating the first variance of the effective fundamental frequency elements in the fundamental frequency matrix after updating the matrix set each time and extracting all blocks composed of preset detection values from the detection result matrix after singular point smoothing processing when the spectrum matrix is full of columns; counting the number of elements of each block and calculating the second variance of all element numbers; judging whether there is baby crying in the voice signal according to the second variance and a preset second variance threshold; and continuing to update the matrix set when it is determined that there is no baby crying. The application can improve the accuracy of baby crying detection.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of voice detection, in particular to a baby crying detection method and device and electronic equipment. BACKGROUND

[0002] With the development of society, problems such as high labor cost gradually emerge, and the cost or time cost of baby care is getting higher and higher. Baby care products or home security products that can recognize baby crying, such as baby care devices, are more and more favored by parents. When the baby care product or home security product detects baby crying, it automatically sends a warning to the caregiver or parent so as to take care of the baby in time.

[0003] In the prior art, when detecting baby crying, it is usually determined whether there is baby crying according to the intensity of the detected voice signal. When the intensity of the detected voice signal is greater than a certain fixed threshold, it is determined that there is baby crying in the voice signal, and a warning is sent to the caregiver or parent.

[0004] However, the detection environment is often complex, and there may be other sounds in the surrounding environment. The voice signal may contain environmental noise that is not baby crying, and the intensity of the other sound is the same as that of the baby crying. If only the intensity of the voice signal is used to detect baby crying, it is likely that environmental noise will be detected as baby crying, and the detection accuracy is low. SUMMARY

[0005] The technical problem to be solved by the embodiments of the present application is to provide a baby crying detection method, device and electronic equipment, which can improve the accuracy of baby crying detection.

[0006] In order to solve the above technical problems, in a first aspect, the present application provides a baby crying detection method, comprising:

[0007] When a new frame of voice signal is obtained according to a preset audio frame length, the frequency spectrum, fundamental frequency and voice activity detection result corresponding to the voice signal are calculated; wherein the fundamental frequency includes invalid fundamental frequency and valid fundamental frequency;

[0008] According to the frequency spectrum, the fundamental frequency and the voice activity detection result corresponding to the new frame of voice signal, the matrix set established in advance is updated according to the first-in first-out principle; wherein the matrix set includes a frequency spectrum matrix, a fundamental frequency matrix and a detection result matrix, and the number of columns of each matrix is M, and M>1;

[0009] After updating the matrix set each time, and when the frequency spectrum matrix is full of columns, the first variance of the valid fundamental frequency elements in the fundamental frequency matrix is calculated, and it is judged whether the first variance is greater than a preset first variance threshold;

[0010] If the first variance is not greater than the first variance threshold, extracting each block composed of a preset detection value from the detection result matrix after the singular point smoothing processing, counting the number of elements of each block, and calculating the second variance of all the element numbers;

[0011] According to the second variance and a preset second variance threshold, it is judged whether there is baby crying in the current M frames of the voice signal; and when it is determined that there is no baby crying in the current M frames of the voice signal, the matrix set is continuously updated.

[0012] Further, the calculation of the spectrum, the fundamental frequency and the voice activity detection result corresponding to the voice signal specifically includes:

[0013] When a new frame of the voice signal is obtained, the spectrum corresponding to the voice signal is obtained based on the Fourier transform algorithm;

[0014] From the spectrum, a frequency point corresponding to a frequency within a preset frequency range is selected, the energy of each selected frequency point is calculated, and the maximum energy frequency point in all the frequency points is determined; and the frequency corresponding to the maximum energy frequency point is taken as the fundamental frequency corresponding to the voice signal; wherein the energy is the sum of the fundamental frequency energy and the harmonic energy of the frequency point;

[0015] The voice signal is detected based on a voice activity detection algorithm to obtain a voice activity detection result corresponding to the voice signal; wherein different voice activity detection results correspond to different detection values.

[0016] Further, after obtaining the voice activity detection result corresponding to the voice signal, the method further includes:

[0017] According to the voice activity detection result corresponding to the voice signal, it is judged whether the fundamental frequency corresponding to the voice signal needs to be invalidated;

[0018] When it is determined that the fundamental frequency corresponding to the voice signal needs to be invalidated, the fundamental frequency corresponding to the voice signal is corrected to the invalid fundamental frequency.

[0019] Further, before selecting the frequency point corresponding to the frequency within the preset frequency range from the spectrum, the method further includes:

[0020] The spectrum is sequentially subjected to fast Fourier transform, low-pass filtering processing and inverse fast Fourier transform to smooth the spectrum; or,

[0021] The spectrum is smoothed based on a preset window smoothing algorithm.

[0022] Further, the preset frequency range is 250Hz-600Hz.

[0023] Further, the method performs singular point smoothing processing on the detection result matrix by the following steps:

[0024] For the i-th detection value in the detection result matrix, if the following two conditions are met simultaneously:

[0025] v(i-r)=v(i+r)

[0026] v(i-r)≠v(i)

[0027] then the i-th detection value in the detection result matrix is smoothed according to the following formula:

[0028] v'(i)=v(i-r)

[0029] wherein, r is a preset smoothing radius, v(i-r) is the i-r-th detection value in the detection result matrix, v(i) is the i-th detection value in the detection result matrix, v(i+r) is the i+r-th detection value in the detection result matrix, and v'(i) is the smoothed detection value of v(i).

[0030] Further, the voice activity detection result includes an active result corresponding to a preset first detection value and an inactive result corresponding to a preset second detection value; then, the preset detection value is the second detection value, and the determining whether the baby cry exists in the M frames of the voice signal according to the second variance and a preset second variance threshold value specifically includes:

[0031] determining whether the second variance is greater than the second variance threshold value;

[0032] if the second variance is not greater than the second variance threshold value, determining that the baby cry exists in the M frames of the voice signal;

[0033] if the second variance is greater than the second variance threshold value, determining that the baby cry does not exist in the M frames of the voice signal.

[0034] Further, each frame of the voice signal includes N time domain points, and N=2 n , n>1; then, the frequency spectrum includes frequency points.

[0035] To solve the corresponding technical problems, the second aspect, the present application also provides a baby cry detection device, the device comprises:

[0036] The first calculation module is configured to calculate a spectrum, a fundamental frequency and a voice activity detection result corresponding to a new frame of voice signal according to a preset audio frame length;

[0037] The updating module is configured to update a matrix set previously established according to the spectrum, the fundamental frequency and the voice activity detection result corresponding to the new frame of voice signal according to a first-in first-out principle; wherein the matrix set comprises a spectrum matrix, a fundamental frequency matrix and a detection result matrix, and the number of columns of each matrix is M, and M>1;

[0038] The first judgment module is configured to calculate a first variance of effective fundamental frequency elements in the fundamental frequency matrix after each update of the matrix set, and determine whether the first variance is greater than a preset first variance threshold value when the spectrum matrix is full of columns;

[0039] The second calculation module is configured to extract each block composed of a preset detection value from the detection result matrix after singular point smoothing processing if the first variance is not greater than the first variance threshold value, count the number of elements of each block, and calculate a second variance of all the element numbers.

[0040] The second judgment module is configured to determine whether there is baby crying in the current M frames of voice signal according to the second variance and a preset second variance threshold value, and continue to update the matrix set when it is determined that there is no baby crying in the current M frames of voice signal.

[0041] To solve the corresponding technical problems, in a third aspect, the present application further provides an electronic device, comprising a processor, a memory and a computer program stored in the memory and configured to be executed by the processor, wherein the processor executes the computer program to implement any one of the baby crying detection methods provided in the first aspect.

[0042] The baby crying detection method, device and electronic device provided above can consider the frequency variation characteristics and periodicity of baby crying, and reflect the frequency variation characteristics through the first variance of the fundamental frequency corresponding to the voice signal, and reflect the periodicity through the second variance of the number of preset detection values in the block of the activity detection matrix, so as to detect the baby crying in line with the sound characteristics of baby crying. Since the frequency variation characteristics and periodicity of baby crying are not easy to be imitated or confused, compared with the technical solution of detecting baby crying by sound intensity in the prior art, the present application can improve the accuracy of baby crying detection. BRIEF DESCRIPTION OF DRAWINGS

[0043] Figure 1 is a flowchart of a preferred embodiment of the baby crying detection method provided by the present application.

[0044] Figure 2 is a structural schematic diagram of a preferred embodiment of the infant cry detection device provided by the present application;

[0045] Figure 3 is a structural schematic diagram of a preferred embodiment of the electronic device provided by the present application. DETAILED DESCRIPTION

[0046] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0047] The present application provides an infant cry detection method, please refer to Figure 1 , Figure 1 is a flowchart of a preferred embodiment of the infant cry detection method provided by the present application. Specifically, the method comprises:

[0048] S1, each time a new frame of speech signal is obtained according to a preset audio frame length, the spectrum, fundamental frequency and voice activity detection result corresponding to the speech signal are calculated; wherein the fundamental frequency includes invalid fundamental frequency and valid fundamental frequency;

[0049] S2, according to the spectrum, the fundamental frequency and the voice activity detection result corresponding to the new frame of speech signal, the matrix set established in advance is updated according to the first-in first-out principle; wherein the matrix set includes a spectrum matrix, a fundamental frequency matrix and a detection result matrix, and the number of columns of each matrix is M, and M>1;

[0050] S3, after updating the matrix set each time, and when the spectrum matrix is full of columns, the first variance of the valid fundamental frequency elements in the fundamental frequency matrix is calculated, and it is judged whether the first variance is greater than a preset first variance threshold;

[0051] S4, if the first variance is not greater than the first variance threshold, each block composed of a preset detection value is extracted from the detection result matrix after singular point smoothing processing, the number of elements of each block is counted, and the second variance of all the element numbers is calculated;

[0052] S5, according to the second variance and a preset second variance threshold, it is judged whether there is infant cry in the current M frames of speech signal; and when it is determined that there is no infant cry in the current M frames of speech signal, the matrix set is continuously updated.

[0053] In implementation, the ambient sound is monitored, and the monitored sound is divided into each frame of speech signal according to a certain audio frame length. There can be overlap between frames or no overlap. After obtaining a new frame of speech signal, the spectrum, fundamental frequency and voice activity detection result of the new frame of speech signal are calculated. The voice activity detection result can be obtained according to the existing related technology for detecting whether the sound is active. Generally, the environmental sound is considered as non-active sound, and the corresponding voice activity detection result is non-active result. The non-environmental sound (such as human speech sound and baby crying sound) is considered as active sound, and the corresponding voice activity detection result is active result. The fundamental frequency is calculated according to the spectrum. The fundamental frequency includes invalid fundamental frequency and valid fundamental frequency. The fundamental frequency corresponding to a frame of speech signal can only be invalid fundamental frequency or valid fundamental frequency.

[0054] When the spectrum matrix, the fundamental frequency matrix and the detection result matrix of the matrix set are updated, it is indicated that the spectrum, the fundamental frequency and the voice activity detection result of a new frame of speech signal are inserted. If the spectrum matrix is not full column, it is indicated that the total number of obtained speech signals has not reached M frames. Then, a new frame of speech signal is continuously obtained to update the matrix set. If the spectrum matrix is full column, it is indicated that the total number of obtained speech signals reaches M frames or more than M frames. The information of M frames of speech signals stored in the spectrum matrix, the fundamental frequency matrix and the detection result matrix is processed. The inventor of the present application finds that the fundamental frequency change range of baby crying sound is small, and the fundamental frequency change range of speech sound, music sound or sudden sound of ambient environment is large. Therefore, the variance of the fundamental frequency is used to preliminarily screen the sound. Specifically, the first variance of the valid fundamental frequency element in the fundamental frequency matrix is calculated, and whether the first variance is greater than a first variance threshold is judged. If the first variance is greater than the first variance threshold, it is indicated that there is no baby crying sound in the M frames of speech signals in the matrix set. Then, a new frame of speech signal is continuously obtained to update the matrix set. If the first variance is not greater than the first variance threshold, it is preliminarily judged that the M frames of speech signals can have baby crying sound.

[0055] Further, the inventor of the present application found that the baby's crying has the characteristics of periodic change during the invention, so the present application further detects the periodicity of the voice signal preliminarily judged as baby's crying: extracting each block composed of a preset detection value from the detection result matrix after the singular point smoothing processing, the preset detection value is the detection value corresponding to a certain voice activity detection result, the block composed of the preset detection value includes a plurality of preset detection values, the number of elements (i.e. the preset detection value) of the block indicates the number of continuous frames of the voice activity detection result, when the number of continuous frames of the voice activity detection result is small, it is very likely that there is baby's crying in the M frame voice signal, so the number of elements of each block is counted, and the second variance of all the element numbers is calculated to reflect the periodicity of the voice signal through the second variance. According to the second variance and the preset second variance threshold, it is judged whether there is baby's crying in the current M frame voice signal. And when it is determined that there is no baby's crying in the current M frame voice signal, an alarm information can be generated and sent to the caregiver or the parent, or the current M frame voice signal can be input into the subsequent accurate detection model as the preliminary detection of baby's crying for further high-precision detection; when it is determined that there is no baby's crying in the current M frame voice signal, the matrix set is updated with the new voice signal for detection and judgment.

[0056] The baby's crying detection method provided by the present application can consider the frequency variation characteristics and periodicity of baby's crying, and reflect the frequency variation characteristics through the first variance of the fundamental frequency corresponding to the voice signal, and reflect the periodicity through the second variance of the number of preset detection values in the block of the activity detection matrix, which can detect baby's crying by matching the sound characteristics of baby's crying. Since the frequency variation characteristics and periodicity of baby's crying are not easy to be imitated or confused, compared with the technical solution of detecting baby's crying by sound intensity in the prior art, the present application can improve the accuracy of baby's crying detection.

[0057] It should be noted that the frequency spectrum matrix is a matrix for storing the frequency spectrum corresponding to the M frames of voice signals, the fundamental frequency matrix is a matrix for storing the fundamental frequency corresponding to the M frames of voice signals, and the detection result matrix is a matrix for storing the voice activity detection result corresponding to the M frames of voice signals. The column number of the frequency spectrum matrix, the fundamental frequency matrix, and the detection result matrix is M, and each column corresponds to a frame of voice signal. It should be noted that since the frequency spectrum includes a plurality of frequency points, the row number of the frequency spectrum matrix also includes a plurality of rows, for example, the element of the first column and the first row of the frequency spectrum matrix corresponds to the frequency information of the first frequency point of the first frame of voice signal. The fundamental frequency and the voice activity detection result of each frame of voice signal are only one, so the fundamental frequency matrix and the detection result matrix are both one row and M columns. It should be noted that the rows and columns of the matrix can be transformed, so the frequency spectrum matrix, the fundamental frequency matrix, and the detection result matrix of the present application can also be correspondingly set to M rows, each row corresponding to a frame of voice signal. Alternatively, each matrix of the present application can also be replaced by an array, and the array can represent the corresponding information by using various symbols, numbers, etc. These alternative schemes are within the protection scope of the present application.

[0058] It should be noted that the first-in first-out principle means that when the information of a new frame of voice signal needs to be stored, the information of each frame of voice signal stored in the matrix is moved one column towards the head end. At this time, the information of the new frame of voice signal will be inserted from the tail end of the corresponding matrix. If the original matrix is full of columns, the voice signal stored in the matrix for the longest time will be removed from the matrix. If the original matrix is not full of columns, the voice signal stored in the matrix for the longest time will not be removed from the matrix, but will only be moved one column towards the head end. It should be noted that the frequency spectrum matrix, the fundamental frequency matrix, and the detection result matrix are all empty matrices when initialized. With the acquisition of voice signals, the frequency spectrum matrix, the fundamental frequency matrix, and the detection result matrix will be gradually updated and filled.

[0059] It should be noted that the voice activity detection result includes active results and inactive results, which correspond to different detection values and are used to indicate whether the voice signal is active sound. Preferably, the duration of the inactive sound is detected, that is, the preset detection value is preferably the detection value corresponding to the inactive result.

[0060] Alternatively, the audio frame length is between 10 ms and 30 ms; in the voice activity detection result, the first detection value corresponding to the active result is 1, and the second detection value corresponding to the inactive result is 0; the value of the invalid fundamental frequency can be represented by 0.

[0061] Preferably, the calculation of the frequency spectrum, the fundamental frequency, and the voice activity detection result corresponding to the voice signal specifically comprises:

[0062] When a new frame of the voice signal is acquired, the frequency spectrum corresponding to the voice signal is obtained based on the Fourier transform algorithm;

[0063] selecting a frequency point corresponding to a frequency within a preset frequency range from the spectrum, calculating an energy of each of the selected frequency points, determining a maximum energy frequency point among all the frequency points, and taking a frequency corresponding to the maximum energy frequency point as a fundamental frequency corresponding to the voice signal; wherein the energy is a sum of a fundamental frequency energy and a harmonic energy of the frequency point;

[0064] detecting the voice signal based on a voice activity detection algorithm to obtain a voice activity detection result corresponding to the voice signal; wherein different voice activity detection results correspond to different detection values.

[0065] Specifically, in the calculation of the spectrum, the fundamental frequency and the voice activity detection result corresponding to the voice signal, the spectrum is obtained by a Fourier transform algorithm, which can be a fast Fourier transform algorithm (FFT) or a discrete Fourier transform algorithm (DFT). The frequency points within the preset range are selected from the spectrum, the energy of the selected frequency points is calculated, and the maximum energy frequency point with the maximum energy is determined. The frequency corresponding to the maximum energy frequency point is taken as the fundamental frequency corresponding to the voice signal. As for the voice activity detection result, a voice activity detection algorithm (VAD) is used to obtain it. The Fourier transform algorithm, the frequency point energy calculation and the VAD algorithm are relatively mature technologies at present, which can be realized by those skilled in the art, and will not be described here.

[0066] It should be noted that the voice activity detection algorithm (VAD) is to distinguish as much as possible between inactive sound (environmental sound) and sound to be identified (in this application, baby crying). Optionally, the environmental sound is labeled as "0", and the baby crying is labeled as "1".

[0067] Preferably, the preset frequency range is a specific frequency range of baby crying, which facilitates the preliminary filtering of frequency points that are not baby crying, reduces the calculation time and improves the detection efficiency.

[0068] Preferably, after obtaining the voice activity detection result corresponding to the voice signal, the method further comprises:

[0069] determining whether the fundamental frequency corresponding to the voice signal needs to be invalidated according to the voice activity detection result corresponding to the voice signal;

[0070] When it is determined that the fundamental frequency corresponding to the voice signal needs to be invalidated, the fundamental frequency corresponding to the voice signal is corrected to the invalid fundamental frequency.

[0071] Specifically, the voice signal is continuously detected, and there can be a voice signal with no sound, and the fundamental frequency of such voice signal is 0, which is considered as an invalid fundamental frequency. There can also be non-active sound, such as environmental sound. In order to accurately determine the fundamental frequency variation characteristics of the voice signal and prevent the frequency variation of some environmental sound from interfering with the detection of the baby's crying sound, the fundamental frequency of the voice signal corresponding to the non-active result is invalidated and corrected to an invalid fundamental frequency. If the invalid fundamental frequency is directly represented by 0, the fundamental frequency of the voice signal corresponding to the non-active result is corrected from the originally calculated fundamental frequency to 0. It should be noted that after the fundamental frequency corresponding to the voice signal is corrected, the originally calculated fundamental frequency is replaced by the invalid fundamental frequency, and the fundamental frequency corresponding to the voice signal used subsequently is all the replaced invalid fundamental frequency.

[0072] Preferably, before selecting the frequency points with the corresponding frequency in the preset frequency range from the spectrum, the method further comprises:

[0073] performing fast Fourier transform, low-pass filtering processing and inverse fast Fourier transform on the spectrum in sequence to smooth the spectrum; or,

[0074] smoothing the spectrum based on a preset window smoothing algorithm.

[0075] Specifically, considering that the detected voice signal is a time domain signal, when transformed into a frequency domain signal to obtain a spectrum, it is possible that due to some noise or interference, the spectrum is discontinuous and the spectrum curve is zigzag-shaped, which affects the quality of the spectrum and directly affects the accuracy of the spectrum analysis. Therefore, after obtaining the spectrum, before selecting the frequency points, the spectrum needs to be smoothed. Specifically, the spectrum is subjected to fast Fourier transform, the signal after fast Fourier transform is subjected to low-pass filtering processing, and the signal after low-pass filtering processing is further subjected to inverse fast Fourier transform to obtain a smoothed spectrum. Alternatively, a window smoothing algorithm is directly used to smooth the spectrum according to a preset window length. Assuming that the window length L = 3 during window smoothing, the a-th frequency point in the spectrum is smoothed according to the formula: X'(a) = (X(a-1) + X(a) + X(a+1)) / 3, where X'(a) is the amplitude of the a-th frequency point after smoothing, X(a-1) is the amplitude of the a-1-th frequency point, X(a) is the amplitude of the a-th frequency point, and X(a+1) is the amplitude of the a+1-th frequency point. It should be noted that for the edge frequency points in the spectrum, an existing edge extension method can be used to extend the edge frequency points to ensure the correctness of the smoothing calculation.

[0076] Preferably, the preset frequency range is 250Hz-600Hz.

[0077] Preferably, the method performs singular point smoothing processing on the detection result matrix by the following steps:

[0078] For the i-th detection value in the detection result matrix, if the following two conditions are met simultaneously:

[0079] v(i-r)=v(i+r)

[0080] v(i-r)≠v(i)

[0081] then the i-th detection value in the detection result matrix is smoothed according to the following formula:

[0082] v'(i)=v(i-r)

[0083] wherein r is a preset smoothing radius, v(i-r) is the i-r-th detection value in the detection result matrix, v(i) is the i-th detection value in the detection result matrix, v(i+r) is the i+r-th detection value in the detection result matrix, and v'(i) is the smoothed detection value of v(i).

[0084] Specifically, assuming that the smoothing radius r=1, the active result is 1, and the inactive result is 0, and taking the detection result matrix [1, 0, 1] as an example, v(1)=1, v(2)=0, and v(3)=1. For v(2), since v(1)=v(3) and v(1)≠v(2) are met simultaneously, v(2) is subjected to singular point smoothing, v'(2)=v(1) is obtained, and the detection result matrix after singular point smoothing processing becomes [1, 1, 1]. Similarly, taking the detection result matrix [1, 0, 0] as an example, for v(2), since v(1)≠v(3) is not met, the smoothed detection result matrix is still [1, 0, 0].

[0085] It should be noted that the smoothing radius can be set according to specific conditions, and the above smoothing radius of 1 is only illustrative. The value of the smoothing radius is preferably in the range of 1-3.

[0086] Preferably, the voice activity detection result includes an active result corresponding to a preset first detection value and an inactive result corresponding to a preset second detection value; then, the preset detection value is the second detection value, and the determining whether the baby cry exists in the M frames of the voice signal according to the second variance and a preset second variance threshold specifically includes:

[0087] determining whether the second variance is greater than the second variance threshold;

[0088] if the second variance is not greater than the second variance threshold, it is determined that the baby cry exists in the M frames of the voice signal;

[0089] if the second variance is greater than the second variance threshold, it is determined that the baby cry does not exist in the M frames of the voice signal.

[0090] Specifically, in the embodiment, the second detection value corresponding to the inactive result is set as the preset detection value, assuming that the active result is 1 and the inactive result is 0, assuming that the detection result matrix after the singular point smoothing processing is [1, 1, 0, 0, 1, 1, 0, 0, 0, 0, 1, 1, 1, 0, 0, 0, 1, 1], the blocks composed of 0 are selected: [0, 0], [0, 0, 0, 0] and [0, 0, 0], the element numbers of the three blocks are 2, 4 and 3 respectively, all the element numbers form an array V = [2, 4, 3], the second variance of all elements in V is calculated. It is judged whether the second variance is greater than the second variance threshold; if the second variance is not greater than the second variance threshold, it indicates that the current M frame voice signal is periodic, and it is determined that the current M frame voice signal contains baby crying; if the second variance is greater than the second variance threshold, it indicates that the current M frame voice signal is not periodic, and it is determined that the current M frame voice signal does not contain baby crying.

[0091] It should be noted that, considering that when performing voice activity detection, sounds that are not baby crying may also be detected as active sounds and marked with "1", for example, siren sounds with specific intonation, or other sounds emitted by babies, or sounds of pets such as cats, and the like. These active sounds have different durations and unstable periodicity, therefore, the preset detection value is preferably set as the second detection value, and the second variance of the element numbers of each block corresponding to the inactive result (environmental sound) is calculated, that is, the periodicity of the voice signal is indirectly reflected through the periodicity of the inactive sound. In actual application, the preset detection value can also be set as the first detection value according to actual needs or the accuracy of voice activity detection, that is, the second variance of the element numbers of the blocks corresponding to the active result is calculated.

[0092] Preferably, each frame of the voice signal includes N time domain points, N = 2 n , n > 1; then, the frequency spectrum includes frequency points.

[0093] Specifically, the frequency spectrum has symmetry, and only N / 2+1 frequency points of the frequency spectrum are considered when the frequency spectrum is obtained according to Fourier transform calculation of the N time domain points. For example, each frame of voice signal has 128 time domain points, i.e. N = 128, and the frequency domain has 65 valid frequency points. It should be noted that the frequency spectrum used in the subsequent steps is the frequency spectrum after considering the symmetry of the frequency spectrum.

[0094] The infant crying detection method provided by the application, in specific implementation, the surrounding sound is monitored, the monitored sound is divided into each frame of speech signal according to a certain audio frame length, there can be overlap between frames or there can be no overlap; each time a new frame of speech signal is obtained, the spectrum, the fundamental frequency and the voice activity detection result of the new frame of speech signal are calculated. After the spectrum matrix, the fundamental frequency matrix and the detection result matrix of the matrix set are updated respectively, if the spectrum matrix is not full column, a new frame of speech signal is continuously obtained to update the matrix set; if the spectrum matrix is full column, the information of the M frames of speech signal stored in the spectrum matrix, the fundamental frequency matrix and the detection result matrix is analyzed and processed. Specifically, the application preliminarily screens the sound through the variance of the fundamental frequency, calculates the first variance of the effective fundamental frequency element in the fundamental frequency matrix, and judges whether the first variance is greater than the first variance threshold. If the first variance is greater than the first variance threshold, a new frame of speech signal is continuously obtained to update the matrix set; if the first variance is not greater than the first variance threshold, it is preliminarily judged that the current M frames of speech signal may exist infant crying. Further, the inventor of the application found that the infant crying has the characteristics of periodic change during the invention, so the application further detects the periodicity of the speech signal preliminarily judged as infant crying: each block composed of a preset detection value is extracted from the detection result matrix after the singular point smoothing processing, the preset detection value is a detection value corresponding to a certain voice activity detection result, the block composed of the preset detection value includes a plurality of preset detection values, the number of elements of each block is counted, and the second variance of all the element numbers is calculated to reflect the periodicity of the speech signal through the second variance. According to the second variance and the preset second variance threshold, it is judged whether the current M frames of the speech signal exist infant crying. When it is judged that the current M frames of the speech signal do not exist infant crying, warning information can be generated and sent to the caregiver or the parent, or the current M frames of the speech signal can be input into the subsequent accurate detection model as the infant crying preliminarily detected for further high-precision detection; when it is judged that the current M frames of the speech signal do not exist the infant crying, the matrix set is continuously updated to detect and judge with the new speech signal.

[0095] The infant crying detection method provided by the application can consider the frequency variation characteristics and periodicity of infant crying, and reflect the frequency variation characteristics through the first variance of the fundamental frequency corresponding to the speech signal, and reflect the periodicity through the second variance of the number of preset detection values in the block of the activity detection matrix, can detect the infant crying by matching the sound characteristics of the infant crying, and since the frequency variation characteristics and periodicity of the infant crying are not easy to be imitated or confused, compared with the technical solution for detecting infant crying by sound intensity in the prior art, the application can improve the accuracy of infant crying detection.

[0096] The embodiment of the present application also provides an infant crying detection device, please refer to Figure 2 , Figure 2 is a structure schematic diagram of a preferred embodiment of the infant crying detection device provided by the present application; specifically, the device comprises:

[0097] a first calculation module 11, configured to calculate a spectrum, a fundamental frequency and a voice activity detection result corresponding to a new frame of voice signal according to a preset audio frame length;

[0098] an updating module 12, configured to update a matrix set established in advance according to the spectrum, the fundamental frequency and the voice activity detection result corresponding to the new frame of voice signal according to a first-in first-out principle; wherein the matrix set comprises a spectrum matrix, a fundamental frequency matrix and a detection result matrix, and the number of columns of each matrix is M, and M>1;

[0099] a first judgment module 13, configured to calculate a first variance of the effective fundamental frequency elements in the fundamental frequency matrix after each update of the matrix set, and judge whether the first variance is greater than a preset first variance threshold value;

[0100] a second calculation module 14, configured to extract each block composed of a preset detection value from the detection result matrix after the singular point smoothing processing, count the number of elements of each block, and calculate a second variance of all the element numbers, if the first variance is not greater than the first variance threshold value;

[0101] a second judgment module 15, configured to judge whether there is infant crying in the current M frames of voice signal according to the second variance and a preset second variance threshold value; and continue to update the matrix set when it is determined that there is no infant crying in the current M frames of voice signal.

[0102] Preferably, when the first calculation module 11 calculates the spectrum, the fundamental frequency and the voice activity detection result corresponding to the voice signal, the first calculation module 11 is specifically configured to:

[0103] obtain the spectrum corresponding to the voice signal based on a Fourier transform algorithm when a new frame of voice signal is obtained;

[0104] select a frequency point corresponding to a frequency within a preset frequency range from the spectrum, calculate the energy of each selected frequency point, determine a maximum energy frequency point in all the frequency points, and take the frequency corresponding to the maximum energy frequency point as the fundamental frequency corresponding to the voice signal; wherein the energy is the sum of the fundamental frequency energy and the harmonic energy of the frequency point;

[0105] The voice signal is detected based on a voice activity detection algorithm to obtain a voice activity detection result corresponding to the voice signal; different voice activity detection results correspond to different detection values.

[0106] Preferably, the device further comprises a correction module configured to:

[0107] After the first calculation module 11 obtains the voice activity detection result corresponding to the voice signal, it is determined whether the fundamental frequency corresponding to the voice signal needs to be invalidated according to the voice activity detection result corresponding to the voice signal.

[0108] When it is determined that the fundamental frequency corresponding to the voice signal needs to be invalidated, the fundamental frequency corresponding to the voice signal is corrected to the invalid fundamental frequency.

[0109] Preferably, before selecting the frequency point corresponding to the frequency within the preset frequency range from the spectrum, the method further comprises a spectrum smoothing module configured to:

[0110] Before the first calculation module 11 selects the frequency point corresponding to the frequency within the preset frequency range from the spectrum, the spectrum is sequentially subjected to fast Fourier transform, low-pass filtering processing and inverse fast Fourier transform to smooth the spectrum; or,

[0111] The spectrum is smoothed based on a preset window smoothing algorithm.

[0112] Preferably, the preset frequency range is 250Hz-600Hz.

[0113] Preferably, the device comprises a singular point smoothing module configured to perform singular point smoothing processing on the detection result matrix, and the singular point smoothing module is specifically configured to:

[0114] For the ith detection value in the detection result matrix, if the following two conditions are met simultaneously:

[0115] v(i-r)=v(i+r)

[0116] v(i-r)≠v(i)

[0117] The ith detection value in the detection result matrix is smoothed according to the following formula:

[0118] v'(i)=v(i-r)

[0119] Wherein, r is a preset smoothing radius, v(i-r) is an i-rth detection value in the detection result matrix, v(i) is an ith detection value in the detection result matrix, v(i+r) is an i+rth detection value in the detection result matrix, and v'(i) is a smoothed detection value of v(i).

[0120] Preferably, the voice activity detection result comprises an active result corresponding to a preset first detection value and an inactive result corresponding to a preset second detection value; and the preset detection value is the second detection value.

[0121] determining whether the second variance is greater than the second variance threshold value;

[0122] if the second variance is not greater than the second variance threshold value, determining that the infant cry exists in the M frames of the voice signal;

[0123] if the second variance is greater than the second variance threshold value, determining that the infant cry does not exist in the M frames of the voice signal.

[0124] Preferably, each frame of the voice signal comprises N time domain points, N = 2 n , n > 1; and the spectrum comprises frequency points.

[0125] The infant cry detection device provided by the embodiment of the present application can consider the frequency variation characteristic and periodicity of the infant cry, and reflect the frequency variation characteristic through the first variance of the fundamental frequency corresponding to the voice signal, and reflect the periodicity through the second variance of the number of preset detection values in the block of the activity detection matrix, so that the infant cry can be detected according to the sound characteristics of the infant cry. Since the frequency variation characteristic and periodicity of the infant cry are not easy to be imitated or confused, compared with the technical solution for detecting the infant cry through the sound intensity in the prior art, the present application can improve the accuracy of infant cry detection.

[0126] It should be noted that the infant cry detection device provided by the embodiment of the present application is used to execute the steps of the infant cry detection method described in the above embodiment, and the working principles and beneficial effects of the two are one-to-one corresponding, so that the description will not be repeated.

[0127] Those skilled in the art can understand that the schematic diagram of the infant crying detection device is only an example of the infant crying detection device, and does not constitute a limitation on the infant crying detection device, and can include more or less components than the diagram, or combine certain components, or different components, for example, the infant crying detection device can also include an input / output device, a network access device, a bus, etc.

[0128] The embodiment of the present application also provides an electronic device, please refer to Figure 3 , Figure 3 is a structural schematic diagram of a preferred embodiment of the electronic device provided by the present application; specifically, the electronic device comprises a processor 10, a memory 20, and a computer program stored in the memory and configured to be executed by the processor, and the processor executes the computer program to realize the infant crying detection method provided by any one of the above embodiments.

[0129] Specifically, the processor and the memory in the electronic device can be one or more, and the electronic device can be an infant care device, a computer, a mobile phone, a tablet, etc. which can perform sound detection.

[0130] The electronic device of the embodiment comprises a processor, a memory, and a computer program stored in the memory and executable on the processor. The processor executes the computer program to realize the steps in the infant crying detection method provided by the above embodiments, for example Figure 1 The step S1 shown in the figure, each time a new frame of voice signal is obtained according to a preset audio frame length, the spectrum, the fundamental frequency and the voice activity detection result corresponding to the voice signal are calculated; wherein the fundamental frequency includes invalid fundamental frequency and valid fundamental frequency. Alternatively, the processor executes the computer program to realize the functions of the modules in the above device embodiments, for example, to realize the first calculation module 11, which is used to calculate the spectrum, the fundamental frequency and the voice activity detection result corresponding to the voice signal each time a new frame of voice signal is obtained according to a preset audio frame length; wherein the fundamental frequency includes invalid fundamental frequency and valid fundamental frequency

[0131] For example, the computer program can be divided into one or more modules / units (such as Figure 3The computer program 1, computer program 2,..., one or more modules / units are stored in the memory and executed by the processor to complete the present application. The one or more modules / units can be a series of computer program instruction segments capable of completing a specific function, which are used to describe the execution process of the computer program in the electronic device. For example, the computer program can be divided into a first calculation module 11, an update module 12, a first judgment module 13, a second calculation module 14 and a second judgment module 15, and the specific functions of each module are as follows:

[0132] The first calculation module 11 is configured to calculate the spectrum, the fundamental frequency and the voice activity detection result corresponding to the new frame of voice signal according to the preset audio frame length; wherein the fundamental frequency includes invalid fundamental frequency and valid fundamental frequency;

[0133] The update module 12 is configured to update the pre-established matrix set according to the spectrum, the fundamental frequency and the voice activity detection result corresponding to the new frame of voice signal according to the first-in first-out principle; wherein the matrix set includes a spectrum matrix, a fundamental frequency matrix and a detection result matrix, and the number of columns of each matrix is M, M>1;

[0134] The first judgment module 13 is configured to calculate the first variance of the valid fundamental frequency elements in the fundamental frequency matrix after each update of the matrix set, and judge whether the first variance is greater than a preset first variance threshold;

[0135] The second calculation module 14 is configured to extract each block composed of a preset detection value from the detection result matrix after the singular point smoothing processing if the first variance is not greater than the first variance threshold, count the number of elements of each block, and calculate the second variance of all the element numbers.

[0136] The second judgment module 15 is configured to judge whether there is baby crying in the current M frames of voice signal according to the second variance and a preset second variance threshold, and continue to update the matrix set when it is determined that there is no baby crying in the current M frames of voice signal.

[0137] The processor can be a central processing unit (CPU), and can also be other general-purpose processors, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor. The processor is a control center of the electronic device, and connects various parts of the electronic device through various interfaces and lines.

[0138] The memory can be used to store the computer program and / or modules, and the processor realizes various functions of the electronic device by running or executing the computer program and / or modules stored in the memory, and calling data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application program required by a function (such as a sound playing function, an image playing function, etc.), etc.; and the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory can include a high-speed random access memory, and can also include a nonvolatile memory, for example, a hard disk, a memory, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, at least one disk storage device, a flash memory device, or other volatile solid-state memory device.

[0139] The modules / units integrated in the electronic device, if realized in the form of software function units and sold or used as independent products, can be stored in a computer readable storage medium. Based on such understanding, all or part of the processes in the infant crying detection method provided by the above-mentioned embodiments can also be completed by a computer program instructing related hardware, and the computer program can be stored in a computer readable storage medium. The computer program can implement the steps of the infant crying detection method provided by any of the above embodiments when executed by a processor. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or some intermediate forms, etc. The computer readable medium can include any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, and software distribution medium, etc. It should be noted that the contents included in the computer readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction, for example, in some jurisdictions, according to legislation and patent practice, the computer readable medium does not include electrical carrier signals and telecommunication signals.

[0140] It should be noted that the above electronic device can include, but is not limited to, a processor, a memory, and those skilled in the art can understand that Figure 3 The structural schematic diagram is only an example of the above-mentioned electronic device, and does not constitute a limitation on the electronic device, and can include more or fewer components than the diagram, or combine certain components, or different components.

[0141] The above is the preferred embodiment of the present application. It should be noted that those skilled in the art can make several improvements and refinements without departing from the principles of the present application, and these improvements and refinements are also considered within the scope of protection of the present application.

Claims

1. A method of detecting infant crying, characterized by, The method comprises: When a new frame of voice signal is obtained according to a preset audio frame length, a spectrum, a fundamental frequency and a voice activity detection result corresponding to the voice signal are calculated; wherein the fundamental frequency comprises an invalid fundamental frequency and a valid fundamental frequency; According to the spectrum, the fundamental frequency and the voice activity detection result corresponding to the new frame of voice signal, a matrix set established in advance is updated according to a first-in first-out principle; wherein the matrix set comprises a spectrum matrix, a fundamental frequency matrix and a detection result matrix, and the number of columns of each matrix is M, and M>1; After each update of the matrix set, and when the spectrum matrix is full of columns, a first variance of valid fundamental frequency elements in the fundamental frequency matrix is calculated, and whether the first variance is greater than a preset first variance threshold is judged; If the first variance is not greater than the first variance threshold, each block composed of a preset detection value is extracted from the detection result matrix after singular point smoothing processing, the number of elements of each block is counted, and a second variance of all the element numbers is calculated; According to the second variance and a preset second variance threshold, whether there is baby crying in the current M frames of voice signal is judged; and when it is determined that there is no baby crying in the current M frames of voice signal, the matrix set is continuously updated.

2. The infant cry detection method of claim 1, wherein, The calculation of the spectrum, the fundamental frequency and the voice activity detection result corresponding to the voice signal specifically comprises: When a new frame of voice signal is obtained, the spectrum corresponding to the voice signal is obtained based on a Fourier transform algorithm; The frequency points corresponding to the frequency within a preset frequency range are selected from the spectrum, the energy of each selected frequency point is calculated, the maximum energy frequency point in all the frequency points is determined, and the frequency corresponding to the maximum energy frequency point is taken as the fundamental frequency corresponding to the voice signal; wherein the energy is the sum of the fundamental frequency energy and the harmonic energy of the frequency point; The voice signal is detected based on a voice activity detection algorithm to obtain the voice activity detection result corresponding to the voice signal; wherein different voice activity detection results correspond to different detection values.

3. The infant cry detection method of claim 2, wherein, After obtaining the voice activity detection result corresponding to the voice signal, the method further comprises: According to the voice activity detection result corresponding to the voice signal, whether the fundamental frequency corresponding to the voice signal needs to be invalidated is judged; When it is determined that the fundamental frequency corresponding to the voice signal needs to be invalidated, the fundamental frequency corresponding to the voice signal is modified to the invalid fundamental frequency.

4. The infant cry detection method of claim 2, wherein, Before the frequency points corresponding to the frequency within a preset frequency range are selected from the spectrum, the method further comprises: The spectrum is sequentially subjected to fast Fourier transform, low-pass filtering and inverse fast Fourier transform to smooth the spectrum; or The spectrum is smoothed based on a preset window smoothing algorithm.

5. The infant cry detection method of claim 2, wherein, The preset frequency range is 250Hz-600Hz.

6. The infant cry detection method of claim 1, wherein, The method performs singular point smoothing processing on the detection result matrix by the following steps: For the i th detection value in the detection result matrix, if the following two conditions are met simultaneously: v(i-r)=v(i+r) v(i-r)≠v(i) then the ith detection value in the detection result matrix is smoothed according to the following formula: v'(i)=v(i-r) wherein r is a preset smoothing radius, v(i-r) is the ith-r detection value in the detection result matrix, v(i) is the ith detection value in the detection result matrix, v(i+r) is the ith+r detection value in the detection result matrix, and v'(i) is the smoothed detection value of v(i).

7. The infant cry detection method of claim 1, wherein, The voice activity detection result includes an active result corresponding to a preset first detection value and an inactive result corresponding to a preset second detection value; then the preset detection value is the second detection value, and the determining whether the baby cry exists in the M frames of the voice signal according to the second variance and a preset second variance threshold specifically includes: determining whether the second variance is greater than the second variance threshold; if the second variance is not greater than the second variance threshold, determining that the baby cry exists in the M frames of the voice signal; if the second variance is greater than the second variance threshold, determining that the baby cry does not exist in the M frames of the voice signal.

8. The infant cry detection method of claim 1, wherein, Each frame of the speech signal comprises N time domain points, N = 2 n , n > 1; then, the spectrum comprises frequency points.

9. An infant cry detection apparatus, characterized by The device comprises: a first calculation module configured to calculate a spectrum, a fundamental frequency and a voice activity detection result corresponding to a new frame of voice signal each time a new frame of voice signal is obtained according to a preset audio frame length; wherein the fundamental frequency includes an invalid fundamental frequency and a valid fundamental frequency; an updating module configured to update a matrix set established in advance according to the spectrum, the fundamental frequency and the voice activity detection result corresponding to the new frame of voice signal according to a first-in-first-out principle; wherein the matrix set includes a spectrum matrix, a fundamental frequency matrix and a detection result matrix, and the number of columns of each matrix is M, and M>1; a first judgment module configured to calculate a first variance of valid fundamental frequency elements in the fundamental frequency matrix and determine whether the first variance is greater than a preset first variance threshold each time the matrix set is updated and the spectrum matrix is full of columns; a second calculation module configured to extract each block composed of a preset detection value from a detection result matrix subjected to singular point smoothing processing if the first variance is not greater than the first variance threshold, count the number of elements of each block, and calculate a second variance of all the element numbers; a second judgment module configured to determine whether a baby cry exists in the M frames of the voice signal according to the second variance and a preset second variance threshold, and continue to update the matrix set when it is determined that the baby cry does not exist in the M frames of the voice signal.

10. An electronic device, comprising: A computer program product comprising a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor implements the baby cry detection method according to any one of claims 1 to 8 when executing the computer program.

Images