Digital healthcare device capable of detecting baby crying sound

Abstract

A device for detecting a baby crying sound according to the present invention may comprise: an audio unit for receiving an input of ambient sound; and a processor for classifying the input ambient sound into a baby crying sound on the basis of a frequency domain analysis, converting the classified baby crying sound into a Mel spectrogram image, and applying the converted Mel spectrogram image to a predetermined trained convolutional neural network (CNN) model to determine whether the classified baby crying sound is an actual baby crying sound, thereby detecting the baby crying sound.

Description

A digital healthcare device that can detect baby cries

[0001] The present invention relates to a device for detecting a baby's crying sound, and more specifically, to a device capable of detecting a baby's crying sound using a deep learning model as a digital healthcare device.

[0002] Crying babies is a major problem in home care. Without effective monitoring technology, babysitters may have to stay with the baby all day. One solution is to design an intelligent system that can automatically detect baby cries.

[0003] Previously, various methods and devices for detecting a baby's crying existed. For example, a method and device for detecting a baby's crying using a frequency and duration pattern includes a microphone that receives a sound input and generates a signal corresponding to the sound, a voice signal processing unit that converts the signal output by the microphone into digital data, a control module that receives the digital data output by the voice signal processing unit, detects the frequency to determine a crying candidate region, and determines whether or not crying occurs based on the duration of the determined crying candidate region.

[0004] As another example, a baby cry analysis system is directed to detecting a baby's cry by determining emotional information. The computer receives cry data related to the baby's cry, acquires only a specific time period from the cry data, and creates segmented data. The computer extracts multiple sampled data, distinguishes between consonant segment data and vowel segment data based on waveform changes or repetitions in the time domain, extracts vowel segment data corresponding to the extracted vowels, and extracts feature information from the vowel segment data. The crying analysis system then determines the emotional information corresponding to the cry data by comparing the feature information with reference data.

[0005] The existing first baby crying detection method and device detects based on the frequency and duration pattern of the sound coming through the microphone, so it only detects baby crying and has the problem that the detection accuracy is not high due to a lot of ambient noise. In addition, the second baby crying analysis system distinguishes between vowel data and consonant data, extracts feature information from the vowel section data, and compares it with the reference data to determine emotional information about the crying data. However, this is also very vulnerable to ambient noise, so there was a problem that the detection accuracy was greatly reduced.

[0006] Furthermore, existing deep learning-based processes pose resource constraints for practical applications. This functionality is essential for infants and toddlers, who require 24 / 7 care. Therefore, a more lightweight process is needed for baby crying detection.

[0007] However, there has been no research on a device that can accurately detect a baby's crying sound, classify the reason for crying, and perform this process with few resources.

[0008] The technical problem to be achieved in the present invention is to provide a device for detecting a baby's crying sound.

[0009] Another technical task to be achieved in the present invention is to provide a method for detecting a baby's crying sound.

[0010] Another technical problem to be achieved by the present invention is to provide a computer-readable recording medium having recorded thereon a program for executing a method for detecting a baby's crying sound on a computer.

[0011] The technical problems to be achieved in the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned can be clearly understood by a person having ordinary skill in the technical field to which the present invention belongs from the description below.

[0012] In order to achieve the above technical task, a baby crying detection device according to the present invention may include an audio unit for receiving ambient sound; and a processor for classifying a baby crying sound based on analysis of the input ambient sound in a frequency domain, converting the classified baby crying sound into a Mel spectrogram image, and applying the converted Mel spectrogram image to a predetermined learned model to determine whether the classified baby crying sound is an actual baby crying sound, thereby detecting a baby crying sound.

[0013] When classifying the baby's crying sound, the processor may measure the signal-to-noise ratio (SNR) value in the frequency domain of the input ambient sound through a fast Fourier transform (FFT), and classify the sound as a baby's crying sound if the measured SNR value is greater than a predetermined threshold. The processor may classify the sound as noise rather than a baby's crying sound if the measured SNR value is less than the predetermined threshold.

[0014] The above Mel Spectrogram image may correspond to an image in the frequency domain. The processor may detect a baby's cry by determining whether the converted Mel Spectrogram image corresponds to an actual baby's cry based on frequency pattern information for a specific period of time of a sound corresponding to a baby's cry learned from the predetermined learned model.

[0015] The above processor can determine the reason for the baby's crying based on frequency pattern information for a specific period of time for the baby's crying sound determined to be an actual baby's crying sound using the above-described learned model.

[0016] The above baby crying sound detection device may further include a wireless communication unit that transmits a signal indicating that the baby crying sound has been detected to a user's terminal.

[0017] The above-described learned model may include a CNN (Convolutional Neural Network) model.

[0018] In order to achieve the above-described other technical task, a method for detecting a baby's cry according to the present invention may include a step of receiving ambient sound; a step of classifying a baby's cry based on analysis of the input ambient sound in a frequency domain; a step of converting the classified baby's cry into a Mel spectrogram image; and a step of applying the converted Mel spectrogram image to a predetermined model to determine whether the classified baby's cry is an actual baby's cry, thereby detecting the baby's cry.

[0019] In the above method, the step of classifying the baby's crying sound may include the step of measuring the signal-to-noise ratio (SNR) value in the frequency domain of the input ambient sound through a fast Fourier transform (FFT), and then classifying the baby's crying sound as a baby's crying sound if the measured SNR value is higher than a predetermined threshold. In the above method, the step of classifying the baby's crying sound may further include the step of classifying the baby's crying sound as noise rather than a baby's crying sound if the measured SNR value is below the predetermined threshold.

[0020] In the above method, the step of detecting the baby's crying sound may include a step of detecting the baby's crying sound by determining whether the converted Mel Spectrogram image corresponds to an actual baby's crying sound based on frequency pattern information for a specific time period of the sound corresponding to the baby's crying sound learned from the predetermined learned model.

[0021] In the above method, the step of detecting a baby's crying sound may include a step of determining the reason for the baby's crying based on frequency pattern information of the detected baby's crying sound for a specific period of time using the predetermined learned model.

[0022] The baby crying detection device according to the present invention has the advantage of being easy to apply since it only requires a microphone to input the baby and surrounding sounds to detect the baby crying.

[0023] The present invention has the advantage of being able to detect the part where the crying sound actually starts through sound, reducing resources by converting to the frequency domain rather than the audio file, and developing a process that is lighter in terms of memory and CPU calculations because it does not always run a deep learning model.

[0024] The effects that can be obtained from the present invention are not limited to the effects mentioned above, and other effects not mentioned can be clearly understood by a person having ordinary skill in the art to which the present invention belongs from the description below.

[0025] The accompanying drawings, which are included as part of the detailed description to aid in understanding the present invention, provide embodiments of the present invention and, together with the detailed description, explain the technical idea of ​​the present invention.

[0026] Figure 1 is a diagram illustrating the layer structure of an artificial neural network.

[0027] Figure 2 is a diagram illustrating an example of a deep neural network.

[0028] Figure 3 is a diagram showing a Mel spectrogram of a baby's crying sound.

[0029] FIG. 4 is a block diagram showing a configuration for explaining the function of a baby crying sound detection device according to one embodiment of the present invention.

[0030] FIG. 5 is a diagram illustrating a crying sound prediction flow in a baby crying sound detection device according to one embodiment of the present invention.

[0031] FIG. 6 is a drawing showing the flow of baby crying sound prediction shown in FIG. 5, in which the power spectrum area (550) that classifies everyday life noise by measuring the SNR value by frequency domain and the area (590) that is converted into a mel spectrogram and shown as a mel spectrogram image are enlarged.

[0032] Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. The detailed description set forth below, together with the accompanying drawings, is intended to illustrate exemplary embodiments of the present invention and is not intended to represent the only embodiments in which the present invention may be practiced. The following detailed description includes specific details to provide a thorough understanding of the present invention. However, one of ordinary skill in the art will appreciate that the present invention may be practiced without these specific details.

[0033] The present invention is susceptible to various modifications and embodiments. Specific embodiments are illustrated and described in detail in the drawings. However, this is not intended to limit the present invention to specific embodiments, but rather to encompass all modifications, equivalents, and alternatives falling within the spirit and technical scope of the present invention.

[0034] When a component is referred to as being "connected" or "connected" to another component, it should be understood that it may be directly connected or connected to that other component, but that there may be other components intervening. Conversely, when a component is referred to as being "directly connected" or "connected" to another component, it should be understood that there are no other components intervening.

[0035] Terms such as first, second, etc. may be used to describe various components, but these components should not be limited by these terms. These terms are used solely to distinguish one component from another.

[0036] The terminology used herein is merely used to describe specific embodiments and is not intended to limit the present invention. The singular expression includes the plural expression unless the context clearly indicates otherwise. In this specification, it should be understood that the terms "comprises" or "has" indicate the presence of a feature, number, step, operation, component, part, or combination thereof described in the specification, but do not exclude in advance the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0037] Additionally, terms such as “part,” “unit,” “module,” and “device” described in the specification mean a unit that processes at least one function or operation, which may be implemented by hardware, software, or a combination of hardware and software.

[0038] In some cases, to avoid ambiguity in the concepts of the present invention, well-known structures and devices may be omitted or illustrated in block diagram form focusing on the core functions of each structure and device. Furthermore, the same components are described using the same reference numerals throughout this specification.

[0039] In addition, it is understood that the components of the embodiments described with reference to each drawing are not limited to the specific embodiments, and may be implemented to be included in other embodiments within the scope in which the technical idea of ​​the present invention is maintained, and that multiple embodiments may be re-implemented as one integrated embodiment even if a separate description is omitted.

[0040] Deep Learning is a type of Artificial Neural Network (ANN) that utilizes the theory of the human neural network. It is a set of machine learning models or algorithms that refer to a deep neural network (DNN) that is structured in a layer structure and has one or more hidden layers (hereinafter referred to as intermediate layers) between the input layer and the output layer. Simply put, Deep Learning can be said to be an artificial neural network with a deep layer.

[0041] Deep neural networks (DNNs) can be considered descendants of artificial neural networks (ANNs), transcending existing limitations and achieving success in areas where numerous AI technologies have previously failed. They are the latest version of ANNs. Looking at the modeling of ANNs based on biological neural networks, biological neurons are modeled as nodes, and connections are modeled as weights (synapses).

[0042] Figure 1 is a diagram illustrating the layer structure of an artificial neural network.

[0043] Just as human biological neurons are interconnected in multiple layers to perform meaningful tasks, individual neurons in artificial neural networks are interconnected through synapses, creating multiple layers where the strength of connections between layers can be updated using weights. This multilayer structure and connection strength are utilized in fields such as learning and cognition.

[0044] Each node is connected by weighted links, and the entire model learns by repeatedly adjusting the weights. Weights are the basic means of long-term memory and express the importance of each node. Simply put, an artificial neural network trains the entire model by initializing these weights and updating and adjusting them with the training data set. After training is complete, when a new input value is received, the appropriate output value is inferred. The learning principle of an artificial neural network can be viewed as a process in which intelligence is formed through the generalization of experience and is performed in a bottom-up manner. In Figure 1, when there are two or more intermediate layers (i.e., 5 to 10), it is considered deep and is called a deep neural network. Learning and inference models achieved through such a deep neural network can be referred to as deep learning.

[0045] While artificial neural networks can perform to some extent with a single intermediate layer (commonly referred to as a hidden layer) beyond the input and output layers, as the problem complexity increases, the number of nodes or layers must be increased. While increasing the number of layers to achieve a multilayered model is effective, its application is limited due to the impossibility of efficient learning and the large computational load required to train the network.

[0046] However, by overcoming these limitations, artificial neural networks have been able to achieve deep structures. This has enabled the construction of complex, highly expressive models, leading to groundbreaking results in diverse fields such as speech recognition, face recognition, object recognition, and character recognition.

[0047] Figure 2 is a diagram illustrating an example of a deep neural network.

[0048] A deep neural network (DNN) is an artificial neural network (ANN) with multiple hidden layers between the input layer and the output layer. It is a collection of machine learning models or algorithms that refer to deep neural networks (DNNs) with one or more hidden layers between the input layer and the output layer. The connections in the neural network are made from the input layer to the hidden layer, and from the hidden layer to the output layer.

[0049] Deep neural networks, like typical artificial neural networks, can model complex non-linear relationships. For example, in a deep neural network architecture for object recognition, each object can be represented as a hierarchical structure of basic image elements. Additional layers can then gradually aggregate features from lower layers. This characteristic of deep neural networks allows them to model complex data with a smaller number of units (nodes) compared to similarly implemented artificial neural networks.

[0050] While previous deep neural networks were typically designed as feed-forward neural networks, recent research has successfully applied deep learning structures to recurrent neural networks (RNNs). For example, deep neural network architectures have been applied to language modeling. Convolutional neural networks (CNNs) have been successfully applied to computer vision, with each successful application well documented. More recently, CNNs have been applied to acoustic modeling for Automatic Speech Recognition (ASR), and are considered more successful than existing models. Deep neural networks can be trained using the standard error backpropagation algorithm. The weights can be updated using stochastic gradient descent using the following equation:

[0051] Figure 3 is a diagram showing a Mel spectrogram of a baby's crying sound.

[0052] Newborn babies express their condition through crying. Monitoring methods are needed to detect their cries and help caregivers respond appropriately. A baby's cries have a unique frequency range. For example, babies cry to express things like "sleepy," "hungry," "in pain," "wanting to burp," and "uncomfortable." Each condition has a different frequency. A baby's cry, which has a fundamental frequency below 500 Hz, tends to be amplified to the 1-3 kHz range. When a baby sobs, the sound is generated below 1 kHz, evoking compassion. When a baby swears, the frequency range of 2-4 kHz, the most sensitive to the human ear, is used to express the baby's feelings more strongly.

[0053] In a Mel Spectrogram, the X-axis represents frequency and the Y-axis represents amplitude. It is a feature extraction technique for analyzing audio whose frequency characteristics change over time. It applies the Mel scale, which reflects the human auditory range.

[0054] Figure 3 shows Mel Spectrograms of baby cries expressing 'sleepy' (310), 'hunger' (320), and 'pain' (330), from the left. Baby cries also differ slightly. When a baby is sick, they cry loudly and shakily. When a baby wants to burp, they cry with a chest-squeezing sound as if air is trapped in their chest. When a baby has a wet diaper or is uncomfortable, they make a 'heh heh' sound that sounds like air mixed with the cry. When a baby is hungry, the cries repeat regularly, and when they are sleepy, the cries are not loud at first and the sound repeatedly gets louder and then quieter. However, the frequency of the cries in each state has the same waveform for children all over the world.

[0055] The human ear cannot distinguish the frequency of a baby's cry. Therefore, to reduce this trial and error, the present invention proposes a method for detecting and classifying baby cries using a deep learning model.

[0056] There have been many attempts to predict baby cries by analyzing the frequency domain or to analyze baby emotions through facial expressions. The frequency range of cries varies as much as the number of babies and the emotions they express, and among the various noises in everyday life, there are sounds with similar frequency components to baby cries. The present invention aims to detect baby cries by learning the patterns and frequency domains of baby cries using a deep learning model using Mel Spectrogram. The present invention measures the signal-to-noise ratio (SNR) value using the frequency domain of baby cries, converting sounds estimated to be baby cries into Mel Spectrogram images, and classifying the baby cries using a convolutional neural network (CNN) (algorithm) model. Through this process, the present invention proposes a process that classifies everyday sounds and cries that can be heard through an audio component (e.g., a microphone) and detects baby cries in real time, enabling guardians to monitor them.

[0057] A Mel Spectrogram is a spectrum converted to Mel units. Mel-scale is a scale conversion function that reflects the threshold for human perception of sound found at these pitches.

[0058] When people perceive speech signals, they do not perceive frequencies on a linear scale. This is because it does not gradually become inaudible as it goes to higher frequencies. Furthermore, people are said to be more sensitive to low frequencies than to high ones. In other words, they are sensitive to changes in the range of 500 to 1000 Hz, but have difficulty perceiving changes in the range of 10,000 to 20,000 Hz. This is why we can view this frequency as a mel scale. Mel(f) = 2595 log(1+f / 700) can be converted to a mel scale.

[0059] FIG. 4 is a block diagram showing a configuration for explaining the function of a baby crying detection device according to one embodiment of the present invention, and FIG. 5 is a diagram showing a crying sound prediction flow in a baby crying detection device according to one embodiment of the present invention.

[0060] Referring to FIG. 4, the baby crying detection device (400) may include an audio unit (410), a processor (420), a wireless communication unit (430), and a display unit (440). The audio unit (410) receives sounds surrounding the baby crying detection device (400). An example of the audio unit (410) is a microphone.

[0061] Referring to FIG. 5, the processor (420) can classify the ambient sound input to the audio unit (410) into a baby cry based on a frequency domain analysis. More specifically, the processor (420) measures the signal-to-noise ratio (SNR) value in the frequency domain of the ambient sound input to the audio unit (410) through a fast Fourier transform (FFT). Then, the processor (420) classifies the ambient sound as a baby cry if the measured signal-to-noise ratio (SNR) value is higher than a predetermined threshold. On the other hand, the processor (410) classifies the ambient sound as a noise rather than a baby cry if the measured SNR value is below the predetermined threshold. As illustrated in area 550 of FIG. 5, the SNR value can be measured by frequency domain to classify everyday life noise.

[0062] The processor (420) converts the classified baby crying sound into a Mel spectrogram image. The processor (420) can detect the baby crying sound by applying the converted Mel spectrogram image to a predetermined learned model (e.g., a Convolutional Neural Network (CNN)) to determine whether the classified baby crying sound is an actual baby crying sound. The converted Mel spectrogram image corresponds to an image in the frequency domain. As shown in area 590 of FIG. 5, the classified baby crying sound can be converted into a Mel spectrogram image. In area 590, Mel spectrogram images are shown for the X-axis time domain and the Y-axis frequency domain.

[0063] FIG. 6 is a drawing showing the flow of baby crying sound prediction shown in FIG. 5, in which the power spectrum area (550) that classifies everyday life noise by measuring the SNR value by frequency domain and the area (590) that is converted into a mel spectrogram and shown as a mel spectrogram image are enlarged.

[0064] The processor (420) can determine whether the converted Mel Spectrogram image corresponds to an actual baby cry based on frequency pattern information for a specific period of time of the sound corresponding to the baby cry learned from a predetermined trained CNN model. Furthermore, the processor (420) can classify the reason for the baby crying from frequency pattern information for a specific period of time of the baby cry detected using a predetermined trained CNN model.

[0065] The baby crying detection method according to the present invention detects the baby's crying based on an image in the frequency domain, and detects the baby's crying using an audio unit (410) (e.g., a microphone) without any special equipment. The baby's crying has its own frequency and has the characteristic of changing over time depending on the type of baby crying. As shown in Fig. 5, the processor (420) converts the (ambient) sound into the frequency domain through FFT, and measures the SNR value by tying together the area where the baby's crying is mainly strong and other areas, and determines that sounds that do not pass the threshold are not the baby's crying but everyday life sounds.

[0066] And the baby's crying sound tends to change in a certain pattern over time. To effectively learn this characteristic, it is converted into a Mel-spectrogram. Then, the convolutional neural network (CNN), which is mainly used in image processing, is used to learn the frequency pattern and detect the crying sound. The processor (420) can classify the baby's crying sound by learning the pattern of the crying sound received during a specific time. Through classification using this rule-based method and deep learning model, the accuracy can be greatly improved and the cost can be greatly reduced.

[0067] By comparing the SNR value using the frequency domain with the threshold, it effectively excludes everyday sounds and greatly reduces the resources used, making it very advantageous for tasks that require real-time measurements for a very long time or all day, such as detecting a baby's crying.

[0068] When a baby's crying sound is detected, the wireless communication unit (430) can transmit a notification to the user's terminal linked to the baby crying sound detection device (400) to notify the detection of the baby's crying sound.

[0069] The display unit (440) may display a display to notify the user of the detection of a baby's cry when the processor (420) detects an actual baby's cry. The display unit (440) may be a flat panel display, such as an LCD, LED, or OLED, that displays a received notification signal on the screen. In addition, the display unit (140) may also include a speaker for audio output to output audio notifying the detection of a baby's cry.

[0070] The baby crying sound detection device (400) according to the present invention has the advantage of being easy to apply as it only requires a microphone to input the baby and surrounding sounds to detect the baby crying sound.

[0071] The following Table 1 is a table showing the accuracy of the baby crying sound detection device (400) according to the present invention.

[0072] precisionrecallTotal accuracyClass 0(Crying)0.830.890.88Class 1(Everyday noise)0.900.71

[0073] The CNN (algorithm) model according to the present invention alone achieved an accuracy of 88%, and because it classifies everyday sounds that have a frequency range significantly different from crying sounds through SNR, it can successfully classify sounds in more cases.

[0074] Research on models that classify babies' emotions through facial images, a method widely studied to date, requires cameras and, due to the inherent movement and body roll of infants and toddlers, makes detection itself difficult. Furthermore, extensive use of video sources is necessary to detect facial expressions.

[0075] However, the present invention can detect the part where the crying sound actually starts through sound, and because it converts to a frequency domain rather than an audio file, it can reduce resources, and because it does not always run a deep learning model, it is possible to develop a process that is lighter in terms of memory and CPU calculations.

[0076] Compared to existing methods for classification through sound, this process is more suitable for application in real situations because it performs primary classification through SNR threshold in the frequency domain.

[0077] Furthermore, it can more effectively classify a baby's cries, helping the baby's caregiver manage their time more effectively. Furthermore, because it only uses the audio component (410) (e.g., a microphone), it is easier to integrate with the baby's peripheral devices. Unlike existing tasks that classify cries from other sounds in daily life, the deep learning model can also learn the type of cries, further assisting the baby's caregiver in making informed decisions. Combining this process with an app on a portable device is expected to reduce the burden on caregivers by enabling them to respond to the baby remotely or take quick action.

[0078] The devices described above may be implemented as hardware components, software components, and / or a combination of hardware components and software components. For example, the devices and components described in the embodiments may be implemented using one or more general-purpose computers or special-purpose computers, such as, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a programmable logic unit (PLU), a microprocessor, or any other device capable of executing instructions and responding to them. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. Furthermore, the processing device may access, store, manipulate, process, and generate data in response to the execution of the software. For ease of understanding, the processing device is sometimes described as being used alone; however, one of ordinary skill in the art will recognize that the processing device may include multiple processing elements and / or multiple types of processing elements. For example, a processing unit may include multiple processors, or a processor and a controller. Other processing configurations, such as parallel processors, are also possible.

[0079] Software may include a computer program, code, instructions, or a combination of one or more of these, and may configure a processing device to perform a desired operation or, independently or collectively, command the processing device. The software and / or data may be permanently or temporarily embodied in any type of machine, component, physical device, virtual equipment, computer storage medium or device, or transmitted signal wave, for interpretation by the processing device or for providing instructions or data to the processing device. The software may also be distributed across network-connected computing devices and stored or executed in a distributed manner. The software and data may be stored on one or more computer-readable recording media.

[0080] The embodiments described above are combinations of components and features of the present invention in a predetermined form. Each component or feature should be considered optional unless explicitly stated otherwise. Each component or feature may be implemented without being combined with other components or features. Furthermore, it is also possible to form an embodiment of the present invention by combining some components and / or features. The order of operations described in the embodiments of the present invention may be changed. Some components or features of one embodiment may be included in another embodiment or may be replaced with corresponding components or features of another embodiment. It is self-evident that claims that do not have an explicit citation relationship in the patent claims may be combined to form an embodiment or may be incorporated as a new claim through a post-application amendment.

[0081] In the present invention, the processor (420) may be implemented by hardware, firmware, software, or a combination thereof. When implementing an embodiment of the present invention using hardware, ASICs (Application Specific Integrated Circuits), DSPs (Digital Signal Processors), DSPDs (Digital Signal Processing Devices), PLDs (Programmable Logic Devices), FPGAs (Field Programmable Gate Arrays), etc. configured to perform the present invention may be provided in the processor (420). The method for detecting a baby's cry according to the present invention may also be implemented as a computer-readable recording medium recording a program for executing the method on a computer.

[0082] It will be apparent to those skilled in the art that the present invention can be embodied in other specific forms without departing from the essential characteristics thereof. Therefore, the above detailed description should not be construed as limiting in any respect, but rather as illustrative. The scope of the present invention should be determined by a reasonable interpretation of the appended claims, and all modifications within the scope of equivalents of the present invention are intended to be included within the scope of the present invention.

[0083] Digital healthcare devices that can detect baby crying have industrial applications, such as remotely monitoring whether a baby is awake and crying in the home.

Claims

1. Audio section that receives ambient sound; and Classify the baby's crying sound based on the frequency domain analysis of the above input ambient sound, Convert the above classified baby cries into Mel spectrogram images, A baby crying detection device, comprising a processor that detects a baby crying sound by applying the above-mentioned converted mel spectrogram image to a predetermined learned model to determine whether the classified baby crying sound is an actual baby crying sound.

2. In paragraph 1, A baby crying detection device, wherein the processor, when classifying the baby crying, measures the signal-to-noise ratio (SNR) value in the frequency domain of the input ambient sound through a fast Fourier transform (FFT), and classifies the baby crying if the measured SNR value is above a predetermined threshold.

3. In paragraph 2, A baby crying detection device, wherein the processor classifies the baby crying as noise rather than a baby crying when the measured SNR value is less than a predetermined threshold.

4. In paragraph 1, The above Mel Spectrogram image corresponds to an image in the frequency domain, a baby crying detection device.

5. In paragraph 1, A baby crying detection device, wherein the processor detects a baby crying sound by determining whether the converted Mel Spectrogram image corresponds to an actual baby crying sound based on frequency pattern information for a specific period of time of a sound corresponding to a baby crying sound learned from the predetermined learned model.

6. In paragraph 1, A baby crying detection device, wherein the processor determines the reason for the baby crying based on frequency pattern information for a specific period of time for the baby crying determined to be an actual baby crying using the predetermined learned model.

7. In paragraph 1, A baby crying detection device further comprising a wireless communication unit that transmits a signal indicating that the baby crying has been detected to a user's terminal.

8. In paragraph 1, A baby crying detection device, wherein the above-described learned model includes a CNN (Convolutional Neural Network) model.

9. Step of receiving ambient sound; A step of classifying a baby's crying sound based on analysis of the input ambient sound in the frequency domain; A step of converting the above classified baby crying sound into a Mel spectrogram image; and A method for detecting a baby's crying sound, comprising a step of applying the above-mentioned converted mel spectrogram image to a predetermined model to determine whether the classified baby's crying sound is an actual baby's crying sound, thereby detecting the baby's crying sound.

10. In paragraph 9, A method for detecting a baby's crying, wherein the step of classifying the baby's crying includes the step of measuring a signal-to-noise ratio (SNR) value in the frequency domain of the input ambient sound through a fast Fourier transform (FFT), and then classifying the baby's crying if the measured SNR value is greater than a predetermined threshold.

11. In paragraph 10, A method for detecting a baby's crying sound, wherein the step of classifying the baby's crying sound further includes a step of classifying the baby's crying sound as noise rather than a baby's crying sound when the measured SNR value is less than a predetermined threshold value.

12. In paragraph 9, A method for detecting a baby's crying sound, wherein the step of detecting the baby's crying sound includes a step of detecting the baby's crying sound by determining whether the converted Mel Spectrogram image corresponds to an actual baby's crying sound based on frequency pattern information for a specific time period of the sound corresponding to the baby's crying sound learned from the predetermined learned model.

13. In paragraph 9, A method for detecting a baby's crying sound, wherein the step of detecting the baby's crying sound includes a step of determining the reason for the baby's crying based on frequency pattern information of the detected baby's crying sound for a specific period of time using the predetermined learned model.

14. A computer-readable recording medium recording a program for executing a baby crying sound detection method described in any one of Articles 8 to 13 on a computer.

Images