Crying sound estimation device, crying sound estimation method, and crying sound estimation program

Abstract

A crying sound estimation device, a crying sound estimation method, and a crying sound estimation program are provided that enable accurate estimation of the cause of an infant's crying, which depends on various factors. [Solution] The device comprises a microphone that captures the crying sounds of an infant, a crying sound learning model storage unit that stores a crying sound learning model, a classification unit that uses voice recognition using the crying sound learning model to classify the captured crying sounds and estimate the cause of the infant's crying, and a display unit that displays the cause, and the crying sound learning model is constructed using previously acquired crying sound data of crying sounds that have been tagged with information about the infant.

Description

Technical Field

[0001] The present disclosure relates to a crying sound estimation device, a crying sound estimation method, and a crying sound estimation program.

Background Art

[0002] Prevention of postpartum depression is important, and if left untreated, it can cause significant social losses. The main factors contributing to childcare stress leading to postpartum depression are divided into four categories: the crying of infants, breastfeeding, isolation, and a sense of helplessness. Among these, dealing with crying occurs most frequently and has a large impact on postpartum depression.

[0003] Patent Documents 1 and 2 disclose devices for presenting the causes and countermeasures for infants' crying. The devices disclosed in Patent Documents 1 and 2 are devices that perform frequency analysis on the crying sound of infants and estimate the emotions of infants such as "sleepy" and "hungry" that cause the crying of infants.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0005] The cause of an infant's crying is often physiological crying, but there is also often crying due to organic physical diseases, which is known to be associated with serious diseases such as heart failure, intussusception, and meningitis. Also, the cause and pattern of crying change according to the developmental stage. Thus, an infant's crying changes depending on many factors. The devices disclosed in Patent Documents 1 and 2 do not take into account such many factors.

[0006] In view of the above problems, the present invention aims to provide a crying sound estimation device, a crying sound estimation method, and a crying sound estimation program that enable accurate estimation of the causes of crying in infants and young children, which depend on various factors. [Means for solving the problem]

[0007] A first aspect of the present invention is a crying sound estimation device comprising: a microphone for acquiring crying sounds of infants; a crying sound learning model storage unit for storing crying sound learning models; a classification unit for estimating the cause of crying by classifying the acquired crying sounds using speech recognition based on the crying sound learning model; and a display unit for displaying the cause. The key feature is that the crying sound learning model is constructed using crying sound data of infants that has been previously acquired and tagged with information about the infant.

[0008] In a first embodiment of the present invention, the classification unit may classify crying into either physiological crying or crying due to organic physical disease.

[0009] In a first aspect of the present invention, the system further includes an input unit for receiving the age of an infant, and the classification unit may estimate the cause from the age and the estimated result.

[0010] In a first aspect of the present invention, the system further comprises an imaging device for acquiring images of infants and toddlers, and an image learning model storage unit for storing an image learning model constructed using image data of images that have been previously acquired and tagged with information about the infants and toddlers. The classification unit may classify the acquired crying sounds using speech recognition with a crying sound learning model and image recognition with an image learning model, and estimate the cause from the image data and the classification result.

[0011] In a first embodiment of the present invention, the system further comprises a vital data acquisition device for acquiring vital data of infants and young children, and the classification unit may estimate the cause from the vital data and the classification result.

[0012] In a first embodiment of the present invention, the vital data may be any of the following: heart rate, oxygen saturation, respiratory rate, crying frequency, crying intensity, bowel sounds, breath sounds, and heart sounds of the infant.

[0013] In a first embodiment of the present invention, the vital data acquisition device may be a sound-receiving sensor installed so as to be in contact with the body of an infant.

[0014] In a first aspect of the present invention, the system further comprises a language input unit that accepts input of a language normally used by a caregiver of an infant, and the classification unit may estimate the cause from the language and the classification result.

[0015] In a first embodiment of the present invention, the system further comprises a countermeasure suggestion unit, which determines an appropriate countermeasure from a countermeasure table based on the cause of crying in infants estimated by the classification unit and transmits it to the display unit.

[0016] In a first embodiment of the present invention, the system further comprises a memory unit, and if the classification unit classifies the crying as being caused by an organic physical disease, it may store the crying sound data in the memory unit and display the medical information on the display unit.

[0017] In a first aspect of the present invention, if the classification unit estimates that the cause of the crying sound requires immediate attention, the action suggestion unit may cause the display unit to display an alert.

[0018] In a first aspect of the present invention, the system further comprises a memory unit, and the classification unit may, after estimating the cause of the infant's crying, store the infant's crying sound data, information identifying the infant, the date and time the crying sound was acquired, the duration of the crying, and the estimated cause in the memory unit.

[0019] A second aspect of the present invention is a crying sound estimation method, which is a crying sound estimation device including a crying sound learning model storage unit that stores a crying sound learning model constructed using crying sound data with tags related to information of an infant acquired in advance. The method includes an acquisition step of acquiring the crying sound data of the infant, an estimation step of classifying the acquired crying sound data by voice recognition using the crying sound learning model to estimate the cause of the infant's crying, and a display step of displaying the cause.

[0020] A third aspect of the present invention is a crying sound estimation program, which is a computer including a crying sound learning model storage unit that stores a crying sound learning model constructed using crying sound data with tags related to information of an infant acquired in advance. The program realizes an acquisition function of acquiring the crying sound data of the infant, an estimation function of classifying the acquired crying sound data by voice recognition using the crying sound learning model to estimate the cause of the infant's crying, and a display function of displaying the cause.

[0021] According to the present invention, it is possible to provide a crying sound estimation device, a crying sound estimation method, and a crying sound estimation program that enable accurate estimation of the cause of an infant's crying depending on various factors.

Brief Description of the Drawings

[0022] [Figure 1] It is a schematic diagram showing an example of a crying sound estimation device according to the first embodiment. [Figure 2] It is a block diagram showing an example of the configuration and functional parts of the crying sound estimation device according to the first embodiment. [Figure 3] It is a schematic diagram showing a state where a countermeasure method for the estimated cause of crying by the crying sound estimation device according to the first embodiment is displayed on the display unit. [Figure 4] It is a schematic diagram showing a state where the next countermeasure method for the estimated cause of crying by the crying sound estimation device according to the first embodiment is displayed on the display unit. [Figure 5] It is a schematic diagram showing an example of a sound collection sensor of a crying sound estimation device according to the second embodiment. [Figure 6] It is a schematic diagram showing another example of the sound collection sensor of the crying sound estimation device according to the second embodiment. [Figure 7] It is a flowchart for explaining the crying sound estimation method according to the first embodiment.

Embodiments for Carrying Out the Invention

[0023] Next, embodiments of the present invention will be described with reference to the drawings. In the description of the drawings according to the embodiments, the same or similar parts are denoted by the same or similar reference numerals. However, it should be noted that the drawings are schematic, and the relationships such as the planar dimensions are different from the actual ones. Therefore, specific dimensions should be determined in consideration of the following description. Also, it is a matter of course that there are parts where the dimensional relationships and ratios are different between the drawings.

[0024] Moreover, the embodiments illustrate devices and methods for embodying the technical idea of the present invention, and the technical idea of the present invention does not specify the configuration, arrangement, layout, etc. of each component as the following. The technical idea of the present invention can be variously modified within the technical scope defined by the claims described in the claims.

[0025] (First Embodiment) The crying sound estimation device according to the present embodiment is a device that acquires the crying sound of the crying infant 1 by a microphone as shown in the example of FIG. 1, estimates the cause of crying based on the acquired crying sound, and presents it. In the example shown in FIG. 1, the crying sound of the infant 1 is acquired by the microphone built in the mobile terminal 2, and the cause estimated by the crying sound estimation device is displayed on the display 3 of the mobile terminal 2.

[0026] Figure 2 shows an example of the configuration and functions of the crying sound estimation device 10 according to this embodiment. The crying sound estimation device 10 shown in Figure 2 includes a CPU 201 for executing various calculations, a ROM 202 for storing processing programs, a RAM 203 for storing data, a storage unit 204 for storing various data and calculation results, an I / O (input / output interface) 205, a microphone 206, a display unit 207, an input unit 209, and the like.

[0027] I / O205 is an interface, buffer, etc., for communication (transmitting and receiving).

[0028] Microphone 206 captures the crying sounds of infants.

[0029] The crying sound estimation device 10 according to this embodiment may also be connected to a keyboard, mouse, or the like for input.

[0030] The crying sound estimation device 10 is a variety of electronic computer (computational resource), such as a mobile terminal, personal computer (PC), mainframe, workstation, or cloud computing system. In the example shown in Figure 1, the crying sound estimation device 10 is a mobile terminal, the microphone 206 is built into the mobile terminal, and the display unit 207 is the display of the mobile terminal. The microphone 206 of the crying sound estimation device 10 according to this embodiment is not limited to the example shown in Figure 1, and may be, for example, a microphone connected externally to a personal computer, or crying sound data acquired in advance by a microphone not connected to a personal computer may be input to a personal computer.

[0031] Furthermore, the block diagram in Figure 2 shows the functional units within the CPU 201. When each functional unit of the CPU 201 is implemented by software, the CPU 201 implements each function by executing instructions from the program, which is the software that implements each function. In detail, it includes a classification unit 208, etc. The memory unit 204 includes a crying sound learning model memory unit 210.

[0032] The crying sound learning model memory unit 210 stores the crying sound learning model. The crying sound learning model is constructed using crying sound data of infants and toddlers that has been previously acquired and tagged with information about the infant.

[0033] The classification unit 208 classifies the acquired crying sounds using speech recognition based on a crying sound learning model and estimates the cause of the crying in infants.

[0034] The display unit 207 displays the cause of the infant's crying estimated by the classification unit 208. In the example shown in Figure 1, for example, the display 3 shows lines such as circles and triangles to indicate that crying sounds are being acquired, and the cause estimated by the classification unit 208 is shown as "hungry," "want to be held," "sleepy," "hot," etc. Also, as will be described later with reference to Figures 3 and 4, appropriate countermeasures according to the cause of crying may be displayed.

[0035] The input unit 209 may accept input of the infant's date of birth. The input unit 209 may also accept input of the infant's age in months, developmental stage, the language used by the caregiver, the infant's gender, height, weight, etc. Furthermore, as will be described later, the input unit 209 may accept the caregiver's judgment on whether the cause of crying estimated by the classification unit 208 was correct or not. The input information about the infant may be stored as infant information in the memory unit 204. When the classification unit 208 classifies the acquired crying sounds and estimates the cause of the infant's crying, it may read the infant information stored in the memory unit 204 and use the infant information when classifying the crying sounds and estimating the cause.

[0036] The crying sound estimation device 10 according to this embodiment will be described below with reference to the example shown in Figure 1. As shown in Figure 1, when an infant 1 cries, the crying sound of the infant 1 is recorded by the microphone 206 of the crying sound estimation device 10. The microphone 206 records the crying sound and generates crying sound data. The microphone 206 transmits the generated crying sound data to the classification unit 208.

[0037] The classification unit 208 classifies the acquired crying sounds using speech recognition with a crying sound learning model. The crying sound learning model is constructed using crying sound data of infants and toddlers that have been previously acquired and tagged with information about the infant, and is stored in the crying sound learning model storage unit 210.

[0038] Crying in infants and toddlers can be classified into two categories: those caused by physiological factors and those caused by organic physical illnesses. Physiological crying accounts for approximately 95% of crying and is a response to hunger, discomfort (e.g., a wet diaper), or separation. When an infant's needs are met through responses such as feeding, diaper changes, or cuddling, physiological crying stops.

[0039] Crying due to organic physical illness accounts for approximately 5 percent of all crying. Generally, among diseases, those in which some abnormality such as inflammation or cancer occurs in the organs themselves are called organic physical illnesses, while other diseases, including those whose cause cannot be identified, are called functional physical illnesses. Therefore, crying caused by diseases whose cause can be identified is called crying due to organic physical illness.

[0040] The training data for constructing the crying sound learning model was generated by pre-recording crying sounds of infants and toddlers, converting the recorded crying sound data into spectrogram format, and then tagging the cause of the crying. The crying sound learning model was constructed by performing deep learning using the generated training data.

[0041] When an infant's crying was physiological, the cause of the crying was tagged when generating training data, and the method used to stop the crying was used as the cause of the crying.

[0042] If an infant's crying is due to an organic physical illness, the crying will not stop even if treated as if it were physiological crying. Therefore, if the crying is suspected to be due to an organic physical illness, such as when it does not stop even when treated as if it were physiological crying, the crying sound should be recorded and the infant should be examined by a pediatrician or other medical professional. If a system for receiving online consultations with pediatricians is already in place, the crying sound can be recorded during the online consultation. Training data was generated by tagging the recorded crying sound data with the doctor's diagnosis and medical record information.

[0043] In the crying sound estimation device 10 according to this embodiment, a crying sound learning model may be constructed by performing deep learning using training data generated when the crying of an infant was physiological crying and training data generated when the crying was due to an organic physical disease. By using the learning model constructed in this way, the classification unit 208 can classify the acquired crying sound into either physiological crying or crying due to an organic physical disease through speech recognition.

[0044] If the classification unit 208 classifies the acquired crying sound as physiological crying, it can further estimate the cause of the crying sound.

[0045] The number of emotional classifications for infants changes with age; immediately after birth there are only two types, pleasant and unpleasant, but by around one year of age, this increases to 18 types, including love and fear. Conventional infant crying sound analysis devices classified infant emotions into about five types and selected the cause of crying from one of these five classified emotions. The crying sound estimation device 10 according to this embodiment classifies infant emotions into 18 types according to age and selects the cause of crying from one of these 18 classified emotions. The crying sound estimation device 10 according to this embodiment may display the selection result on the display unit 207 after selecting the cause of crying from one of the 18 types of emotions. The crying sound estimation device 10 according to this embodiment may display the emotion on the display unit 207 as the selection result. However, if the cause of crying has been estimated, it is preferable for the crying sound estimation device 10 according to this embodiment to display the cause of crying on the display unit 207 as the selection result. For example, if the emotion selected as the cause of crying is unpleasant, and the cause of unpleasantness is hunger, the crying sound estimation device 10 according to this embodiment may display "hunger" on the display unit 207.

[0046] Table 1 shows the types of emotions for each age group classified by the crying sound estimation device 10 according to this embodiment. The emotions shown in Table 1 are the emotions of infants that are added at the corresponding age, and typically, the emotions of infants are added starting from those shown in the upper row of Table 1. For example, infants aged 0-1 month first have the emotions of "pleasure," "discomfort," and "interest," and then infants aged 1-2 months have the emotions of "pleasure," "discomfort," and "interest," in addition to the emotions of "anxiety," "security," and "loneliness."

[0047] [Table 1]

[0048] When an infant's crying was physiological, the caregiver's response to stop the crying was recorded as the cause of the crying. The crying sound was recorded for each instance, and simultaneously, the caregiver selected the infant's emotion from the emotions shown in Table 1, corresponding to their age or developmental stage. The recorded crying sounds were tagged with the emotions selected by the caregiver, generating training data.

[0049] In the crying sound estimation device 10 according to this embodiment, a crying sound learning model may be constructed by performing deep learning using training data generated by classifying the causes of crying in infants into emotions according to their age in months. By using the learning model constructed in this way, the classification unit 208 can use the age of the infant received as input in the input unit 209 to classify the causes of crying in the acquired crying sounds into 18 types of emotions through speech recognition.

[0050] When crying in infants is due to an organic physical illness, the main causes of crying can be classified into cardiovascular, gastrointestinal, infectious, and traumatic. Among these causes, those that may be life-threatening include heart failure, intussusception, volvulus, meningitis, and intracranial hemorrhage due to head trauma. Examples of urgent illnesses that can be predicted from crying include, for example, the crying in an asthma attack is characterized by a wheezing sound due to airway narrowing. Crying in meningitis is characterized by a high-pitched, unstable rhythm in infants due to increased intracranial pressure and severe headache. Crying in suffocation is caused by abnormal vibration of the vocal cords due to respiratory distress and exhibits a higher-than-normal fundamental frequency (f0). In intussusception, crying sounds are sudden and intense, and the intensity and frequency of the crying sounds fluctuate periodically with the contraction and relaxation of the intestines. By capturing these characteristics, the disease can be detected. If the classification unit 208 classifies the acquired crying sounds as crying due to an organic physical disease, the classification unit 208 estimates the cause of the acquired crying sounds. The crying sound estimation device 10 may further suggest things like visiting a pediatrician or acquiring vital data, as will be described later.

[0051] As a specific example of a disease that can be estimated by the crying sound estimation device 10 according to this embodiment, Sudden Infant Death Syndrome (SIDS) will be described. SIDS mainly occurs during nighttime sleep, and is most common between 2 and 6 months of age. The cause of onset is unknown, but it is thought to be related to a decrease in the arousal response when respiration is suppressed for some reason. It accounts for 6.1% of infant deaths, and the incidence in Japan is estimated to be about 1 in 6,000 to 7,000 births. A study comparing the cries of 14 infants who died suddenly of unknown cause at 4 days old with those of a neonatal control group, using nine acoustic characteristics including fundamental frequency (f0), duration, formant frequency, and sound pressure level, revealed that the cries of these infants were lower in f0, longer in duration, lower in formant frequency, and higher in sound pressure level across the entire range. This was demonstrated in several papers ("The Cry Characteristics of an Infant Who Died of the Sudden Infant Death Syndrome" (RH Colton, 1981), "Cry features in siblings of SIDS" (Michael P Robb, 2007), "Sudden Infant Death Syndrome: Cry characteristics" (Michael P Robb, 2013)). Utilizing these results, the cry sound estimation device 10 according to this embodiment can be used to estimate the risk of sudden infant death syndrome from the cries of infants.

[0052] Furthermore, when estimating the risk of sudden infant death syndrome, the accuracy can be improved by using oxygen saturation data. As will be described later, the crying sound estimation device 10 according to this embodiment may acquire vital data of infants, particularly oxygen saturation data, and estimate the cause of crying using oxygen saturation data in addition to crying sound data.

[0053] As described above, the classification of crying into physiological crying or crying due to organic physical disease can be performed immediately after acquiring crying data. However, there are diseases that can only be identified by continuously acquiring crying sounds. The classification unit 208 classifies the crying based on the acquired crying sound data and then stores the crying sound data of infants and toddlers in the memory unit 204 as a crying sound database, for example, along with information identifying the infant or toddler, the date and time the crying sound was acquired, the duration of the crying, and the classification result by the classification unit 208. The classification unit 208 stores the above data in the crying sound database each time an infant or toddler's crying sound is acquired, thereby accumulating crying sound data.

[0054] For example, colic is defined as excessive crying that occurs in infants under 4 months of age, without any identifiable organic cause, and lasting for more than 3 hours a day and more than 3 days a week for more than 3 weeks. The classification unit 208 can classify the acquired crying sounds into physiological crying or organic crying using speech recognition with a crying sound learning model, and also compare them with crying sound data stored in a crying sound database, thereby estimating diseases such as colic that are revealed by continuously acquired crying sound information.

[0055] As described above, abnormal crying in infants and toddlers can be estimated by voice recognition of the crying sound in the crying sound estimation device 10, and it is possible to detect if the crying is abnormal. In particular, the crying sound estimation device 10 in this embodiment uses a technique called TS-VAD (Target Speaker Voice Activity Detection) when detecting abnormal crying sounds. TS-VAD is pre-trained with crying sounds to be detected and detects from the acquired crying sounds when and for how long the crying sound to be detected was heard. The crying sound estimation device 10 in this embodiment can detect abnormal crying sounds using this method and determine from the abnormal crying sound whether the underlying disease is urgent and requires transport by ambulance, etc., or whether it is not urgent and a medical examination is sufficient.

[0056] Furthermore, the crying sound estimation device 10 according to this embodiment can also detect that crying is abnormal by detecting that "crying is different from normal crying." Even if the crying sound estimation device 10 does not detect any abnormalities such as diseases through voice recognition of crying sounds, it is possible that crying may be different from normal due to a disease or other condition. Therefore, detecting abnormal crying enables early intervention and treatment of diseases and other conditions. In addition, information that crying is different from normal can be helpful for doctors in initial diagnosis. Moreover, by accumulating analysis data of crying over a long period, it is possible to detect abnormalities in infants and young children more accurately and identify their causes.

[0057] The classification unit 208 may detect abnormalities in crying by classifying the acquired crying sounds into physiological crying or organic crying using speech recognition with a crying sound learning model, and by comparing them with crying sound data stored in a crying sound database. For example, the classification unit 208 may classify the acquired crying sound data into physiological crying or organic crying, and then detect whether the crying is abnormal by comparing it with crying sound data stored in the crying sound database that has the same or similar classification result as the classification result of the classification unit 208.

[0058] The crying sound estimation device 10 according to this embodiment can visualize the health data of infants by accumulating the crying sounds of infants in a crying sound database. For example, the crying sound estimation device 10 can identify the developmental stage of an infant by comparing the changes over time in the causes of crying classified as physiological crying stored in the infant crying sound database with the types of emotions for each age of the infant shown in Table 1, and display this information on the display unit 207.

[0059] The types of emotions shown in Table 1 for infants and toddlers by age in months represent the average for infants and toddlers. However, if, for example, a delay greater than a predetermined level is observed in the infant's developmental stage identified from the data stored in the crying sound database relative to the infant's actual age in months, language development delay or developmental disorder may be suspected. The crying sound estimation device 10 can detect data in which delays or abnormalities are observed in emotional differentiation. The crying sound estimation device 10 may conduct a medical interview based on the detected data in which delays or abnormalities are observed, for example, by displaying a questionnaire on the display unit 207. Furthermore, it may enable early rehabilitation by displaying information on medical institutions on the display unit 207.

[0060] Respiratory syncytial virus (RSV) infection can be fatal due to sudden death from apnea, respiratory failure from acute bronchiolitis, and acute myocarditis. After infection, RSV initially causes upper respiratory tract inflammation, followed by symptoms such as runny nose and cough. Approximately 30-40% of cases progress to acute bronchiolitis or pneumonia, with symptoms such as rapid breathing and wheezing. In infants, RSV easily spreads to the lower respiratory tract, and infants under 6 months of age are particularly susceptible to severe illness. Furthermore, premature infants and those with underlying conditions such as congenital heart disease have an even higher risk of developing severe illness. Since there is thought to be a correlation between crying sounds and the risk of severe RSV infection, early detection and prevention of severe illness can be achieved by detecting the risk of severe illness in infants using the crying sound estimation device 10 and vital data according to this embodiment.

[0061] The crying sounds of infants and toddlers change depending on the language that the infant or toddler usually hears, i.e., the caregiver's native language, or the language that the caregiver normally uses. Therefore, in the process of generating training data according to the cause of crying and the age of the infant or toddler, as described above, the language that the caregiver normally uses may be added as a tag to generate the training data. In the crying sound estimation device 10 according to this embodiment, a crying sound learning model may be constructed by performing deep learning using the training data generated in this way, and the classification unit 208 can further improve the accuracy of the classification of the cause of crying sounds obtained by speech recognition using this crying sound learning model, using the language that is mainly used by the caregiver of the infant or toddler that is received as input in the input unit 209.

[0062] The crying sound estimation device 10 according to this embodiment may include an imaging device for acquiring images of infants. If the crying sound estimation device 10 is, for example, a mobile terminal as shown in Figure 1, the imaging device may be a video camera built into the mobile terminal, but is not limited to this, and may be, for example, an externally connected camera.

[0063] When acquiring the crying sounds of infants, it is also possible to capture images of the infant crying using an imaging device and acquire image data of the infant. For example, there is a correlation between facial movements and emotions as shown in Table 2, and by using the infant's facial color, expression, and limb movements during crying, in addition to the crying sounds, the accuracy of the cause estimation can be improved.

[0064] [Table 2]

[0065] When classifying acquired crying sounds, the classification unit 208 may perform classification by using speech recognition with a crying sound learning model on crying sound data of infants and image recognition with an image learning model on image data of infants. The crying sound estimation device 10 according to this embodiment may further include an image learning model storage unit. The image learning model is constructed using image data of crying infants that have been previously acquired and tagged with information about the infant, and is stored in the image learning model storage unit.

[0066] When constructing an image learning model, the information about infants and toddlers attached to the image data may include the cause of crying, and may also include at least one of the following: the infant's age in months or developmental stage, or the primary language used by the caregiver.

[0067] The crying sound estimation device 10 according to this embodiment may acquire vital data of infants and use the vital data, in addition to crying sound data, to estimate the cause of crying. Examples of infant vital data include the infant's heart rate, oxygen saturation, respiratory rate, crying frequency, crying intensity, bowel sounds, respiratory sounds, and heart sounds. The infant's vital data is acquired by a vital data acquisition device. The vital data acquisition device is a general device for acquiring the infant's heart rate, oxygen saturation, respiratory rate, crying frequency, crying intensity, bowel sounds, respiratory sounds, and heart sounds, as examples of infant vital data mentioned above, and may be connected to the crying sound estimation device 10, or it may be possible to acquire the infant's vital data and then input only the acquired vital data into the crying sound estimation device 10. In fact, since there is a correlation between frequent burping and illness, frequency has been added. (→This is often seen in gastroesophageal reflux disease in newborns and tic disorders in infants.)

[0068] Figures 5 and 6 show examples of sound-collecting sensors for acquiring vital data of infants. Figure 5 shows a sound-collecting sensor 53 attached to the ventral side of the diaper 52 of infant 51, and Figure 6 shows a sound-collecting sensor 63 attached to the back side of the diaper 62 of infant 61.

[0069] The sound-collecting sensors 53 and 63 shown in Figures 5 and 6 are attached to the diaper by tape, clips, etc., are removable, and can be attached to any position on the diaper. The sound-collecting sensors 53 and 63 shown in Figures 5 and 6 are not limited to the diaper, but may also be attached to clothing, etc. The sound-collecting sensors 53 and 63 are connected wirelessly to a crying sound estimation device 10 (not shown), and the data acquired by the sound-collecting sensors 53 and 63 are transmitted wirelessly to the crying sound estimation device 10. By installing the sound-collecting sensors 53 and 63 so as to be in contact with the body of an infant, heart sounds, respiratory sounds, and bowel sounds can be acquired, and by combining this with symptoms such as diarrhea, vomiting, and difficulty breathing, diseases such as suffocation and intestinal obstruction can be predicted.

[0070] Acquiring vital data is not as easy as recording crying sounds or taking images, for example, because it requires equipment for acquisition and attachment to the infant's body. Therefore, it is acceptable for this method to be used only when an infant's crying is organic and suggests illness or a physical abnormality in the infant.

[0071] The crying sound estimation device 10 according to this embodiment may further include a countermeasure presentation unit and a countermeasure table storage unit that stores a countermeasure table describing the relationship between the cause of crying in infants and countermeasures. The countermeasure presentation unit may present a countermeasure by determining an appropriate countermeasure from the countermeasure table based on the cause of crying in infants estimated by the estimation unit, transmitting it to the display unit 207, and displaying it on the display unit 207.

[0072] If the crying of an infant is physiological, the coping method suggestion unit will display coping methods on the display unit 207 according to the cause of the crying, such as holding the infant, breastfeeding, stroking the head and body, rocking the infant in a bouncer or while being held, giving a pacifier, or breastfeeding while lying down. For example, as shown in Figure 3, if the cause of the crying is hunger, the display unit 207 will display "give milk" as a coping method. If the crying does not stop, the input unit will receive input from the caregiver, etc., stating that the crying has not stopped even after trying the coping methods suggested by the coping method suggestion unit, and the coping method suggestion unit may then suggest a coping method that is presumed to be the next most effective after the previously suggested coping method. As shown in Figure 4, if the crying does not stop even after the display unit 207 indicates that the cause of crying is hunger and suggests the solution "give milk," the cause of crying may be fatigue rather than hunger, and the following solution may be presented.

[0073] The section providing coping strategies will, depending on the cause of the crying, provide information such as local medical care systems, consultation services for childcare and pediatrics, consultation hours and contact information for medical institutions, and, if a system for receiving online consultations with pediatricians is in place, how to access online consultations.

[0074] Furthermore, the coping mechanism may suggest acquiring the infant's vital data if the crying is due to an organic physical illness, or if the crying is physiological but does not stop despite any coping mechanism suggested by the coping mechanism. Additionally, for example, if a vital data acquisition device is connected to the crying sound estimation device 10 and is in a state where vital data can be acquired, the device may prompt the acquisition of the infant's vital data by sending a signal to the vital data acquisition device to start measurement.

[0075] If the classification result performed by the classification unit 208 of the crying sound estimation device 10 according to this embodiment indicates that the cause of the crying sound requires immediate attention, the countermeasure suggestion unit may display an alert on the display unit 207, and may also display information on emergency medical facilities, etc., on the display unit 207 along with the alert.

[0076] The crying sound estimation method according to this embodiment will be explained with reference to the flowchart in Figure 7. In Figure 7, the flowchart is assumed to be performed in a crying sound estimation device that includes a crying sound learning model storage unit that stores a crying sound learning model constructed using crying sound data that has been previously acquired and tagged with information about infants.

[0077] In step S701, data on the crying sounds of infants is acquired (acquisition step).

[0078] In step S702, the acquired crying sound data is classified using speech recognition with a crying sound learning model, and the cause of the infant's crying is estimated (estimation step).

[0079] In step S703, the cause is displayed (display step).

[0080] As stated above, the present invention naturally includes various embodiments and the like that are not described herein. Therefore, the technical scope of the present invention is determined solely by the inventive features relating to the claims that are reasonable based on the above description. [Explanation of Symbols]

[0081] 1. Infants and toddlers 2 Mobile devices 3 displays 10. Crying Sound Estimation Device 201 CPU 202 ROM 203 RAM 204 Storage section 205 I / O (Input / Output Interface) 206 Microphone 207 Display section 208 Classification Department 209 Input section 210 Crying Sound Learning Model Memory Unit 51, 61 Infants 52, 62 diapers 53, 63 Sound collection sensors

Claims

1. A microphone that captures the crying sounds of infants, A crying sound learning model memory unit that stores crying sound learning models, A classification unit that uses the aforementioned crying sound learning model for speech recognition to classify the acquired crying sounds and estimate the cause of the infant's crying. An imaging device for acquiring images of the infant, An image learning model storage unit stores an image learning model constructed using image data of images tagged with information about infants and toddlers, which have been acquired in advance. A display unit that displays the aforementioned cause, A crying sound estimation device comprising: a crying sound learning model constructed using crying sound data of crying sounds tagged with information about infants and toddlers, which has been acquired in advance; a classification unit classifying the acquired crying sounds by speech recognition using the crying sound learning model and image recognition using the image learning model, and estimating the cause from the image data and the estimation results.

2. A microphone that captures the crying sounds of infants, A crying sound learning model memory unit that stores crying sound learning models, A classification unit that uses the aforementioned crying sound learning model for speech recognition to classify the acquired crying sounds and estimate the cause of the infant's crying. A display unit that displays the aforementioned cause, A vital data acquisition device for acquiring the vital data of the infant, A crying sound estimation device comprising: a crying sound learning model constructed using crying sound data of infants and toddlers tagged with information acquired in advance; and a classification unit that estimates the cause from the vital data and the estimation results.

3. A microphone that captures the crying sounds of infants, A crying sound learning model memory unit that stores crying sound learning models, A classification unit that uses the aforementioned crying sound learning model for speech recognition to classify the acquired crying sounds and estimate the cause of the infant's crying. A language input unit that accepts input of the language normally used by the caregiver of the infant, A display unit that displays the aforementioned cause, A crying sound estimation device comprising: a crying sound learning model constructed using crying sound data of crying sounds tagged with information about infants and toddlers, which has been acquired in advance; and a classification unit that estimates the cause from the language and the estimation result.

4. A microphone that captures the crying sounds of infants, A crying sound learning model memory unit that stores crying sound learning models, A classification unit that uses the aforementioned crying sound learning model for speech recognition to classify the acquired crying sounds and estimate the cause of the infant's crying. A display unit that displays the aforementioned cause, Memory unit and, A crying sound estimation device comprising: a crying sound learning model constructed using crying sound data of crying sounds tagged with information about infants and toddlers, the classification unit classifies the crying into either physiological crying or crying due to organic physical disease, and if the crying is classified as crying due to organic physical disease, the crying sound data is stored in the storage unit and the medical information is displayed on the display unit.

5. A microphone that captures the crying sounds of infants, A crying sound learning model memory unit that stores crying sound learning models, A classification unit that uses the aforementioned crying sound learning model for speech recognition to classify the acquired crying sounds and estimate the cause of the infant's crying. A display unit that displays the aforementioned cause, Section presenting solutions, The crying sound estimation device is characterized by comprising: a crying sound learning model constructed using crying sound data of infants and toddlers that have been previously acquired and tagged with information about the infants and toddlers; a countermeasure presentation unit determining an appropriate countermeasure from a countermeasure table based on the cause of the infant's crying estimated by the classification unit and transmitting it to the display unit; and when the classification unit estimates that the cause of the crying sound requires urgent attention, the countermeasure presentation unit causes the display unit to display an alert.

6. The crying sound estimation device according to any one of claims 1 to 3, 5, characterized in that the classification unit classifies the crying into either physiological crying or crying due to organic physical disease.

7. The crying sound estimation device according to any one of claims 1 to 5, further comprising an input unit for receiving the age of the infant, wherein the classification unit estimates the cause from the age and the estimation result.

8. The crying sound estimation device according to claim 2, characterized in that the vital data is at least one of the following: heart rate, oxygen saturation, respiratory rate, crying frequency, crying intensity, bowel sounds, breath sounds, and heart sounds of the infant.

9. The crying sound estimation device according to claim 8, characterized in that the vital data acquisition device is a sound-collecting sensor installed so as to be in contact with the body of the infant.

10. The crying sound estimation device according to any one of claims 1 to 4, further comprising a countermeasure presentation unit, wherein the countermeasure presentation unit determines an appropriate countermeasure from a countermeasure table based on the cause of the crying of the infant estimated by the classification unit and transmits it to the display unit.

11. The crying sound estimation device according to any one of claims 1 to 3 or 5, further comprising a memory unit, wherein the classification unit estimates the cause of the crying of the infant, and then stores the crying sound data of the infant, information identifying the infant, the date and time the crying sound was acquired, the duration of the crying, and the estimated cause in the memory unit.

12. A crying sound estimation device comprising: a crying sound learning model storage unit that stores a crying sound learning model constructed using crying sound data tagged with information about infants and toddlers acquired in advance; and an image learning model storage unit that stores an image learning model constructed using image data tagged with information about infants and toddlers acquired in advance, and a crying sound estimation method in the device, The computer of the crying sound estimation device, A crying sound data acquisition step to obtain crying sound data of infants and toddlers, An estimation step in which the crying sound data acquired is classified by speech recognition using the crying sound learning model described above, and the cause of the crying of the infant is estimated. A display step that shows the cause, The image acquisition step involves obtaining an image of the infant, The estimation step involves classifying the acquired crying sound data using speech recognition with the crying sound learning model and image recognition with the image learning model, and estimating the cause from the image data and the estimation results. A method for estimating crying sounds, characterized by the following:

13. A computer comprising: a crying sound learning model storage unit that stores a crying sound learning model constructed using crying sound data tagged with information about infants and toddlers acquired in advance; and an image learning model storage unit that stores an image learning model constructed using image data tagged with information about infants and toddlers acquired in advance; A function to acquire data on the crying sounds of infants and toddlers, The system includes an estimation function that uses the aforementioned crying sound learning model to classify the acquired crying sound data and estimate the cause of the infant's crying, A display function that shows the aforementioned cause, The image acquisition function for acquiring images of the aforementioned infants and toddlers, The estimation function achieves this by classifying the acquired crying sound data using speech recognition with the crying sound learning model and image recognition with the image learning model, and estimating the cause from the image data and the estimation results. A crying sound estimation program characterized by the following:

14. A crying sound estimation method in a crying sound estimation device, comprising a crying sound learning model storage unit that stores a crying sound learning model constructed using crying sound data to which information about infants and toddlers has been previously acquired and tagged, the following: The computer of the crying sound estimation device, A crying sound data acquisition step to obtain crying sound data of infants and toddlers, An estimation step in which the crying sound data acquired is classified by speech recognition using the crying sound learning model described above, and the cause of the crying of the infant is estimated. A display step that shows the cause, A vital data acquisition step for acquiring the vital data of the infant, A method for estimating crying sounds, characterized in that the estimation step involves performing the above, and estimating the cause from the vital data and the result of the estimation.

15. A computer equipped with a crying sound learning model storage unit that stores a crying sound learning model constructed using crying sound data that has been previously acquired and tagged with information about infants, A crying sound data acquisition step to obtain crying sound data of infants and toddlers, An estimation step in which the crying sound data acquired is classified by speech recognition using the crying sound learning model described above, and the cause of the crying of the infant is estimated. A display step that shows the cause, A vital data acquisition step for acquiring the vital data of the infant, A crying sound estimation program that achieves the above, and the estimation step is characterized in that the cause is estimated from the vital data and the result of the estimation.

16. A crying sound estimation method in a crying sound estimation device, comprising a crying sound learning model storage unit that stores a crying sound learning model constructed using crying sound data to which information about infants and toddlers has been previously acquired and tagged, the following: The computer of the crying sound estimation device, A crying sound data acquisition step to obtain crying sound data of infants and toddlers, An estimation step in which the crying sound data acquired is classified by speech recognition using the crying sound learning model described above, and the cause of the crying of the infant is estimated. A language input step that accepts input of the language normally used by the caregiver of the infant, A display step that shows the cause, A method for estimating crying sounds, characterized in that the estimation step involves performing the estimation step, and estimating the cause from the language and the result of the estimation.

17. A computer equipped with a crying sound learning model storage unit that stores a crying sound learning model constructed using crying sound data that has been previously acquired and tagged with information about infants, A crying sound data acquisition step to obtain crying sound data of infants and toddlers, An estimation step in which the crying sound data acquired is classified by speech recognition using the crying sound learning model described above, and the cause of the crying of the infant is estimated. A language input step that accepts input of the language normally used by the caregiver of the infant, A display step that shows the cause, A crying sound estimation program comprising the above, wherein the estimation step is characterized in that the cause is estimated from the language and the result of the estimation.

18. A crying sound estimation method in a crying sound estimation device comprising: a crying sound learning model storage unit that stores a crying sound learning model constructed using crying sound data to which information about infants and toddlers has been previously acquired and tagged; and a storage unit, The computer of the crying sound estimation device, A crying sound data acquisition step to obtain crying sound data of infants and toddlers, An estimation step in which the crying sound data acquired is classified by speech recognition using the crying sound learning model described above, and the cause of the crying of the infant is estimated. A display step that shows the cause, The image acquisition step involves obtaining an image of the infant, A method for estimating crying sounds, characterized in that the estimation step classifies the crying into either physiological crying or crying due to an organic physical disease, stores the crying sound data in the storage unit if the crying is classified as crying due to an organic physical disease, and the display step displays medical information.

19. A computer comprising a crying sound learning model storage unit that stores a crying sound learning model constructed using crying sound data tagged with information about infants and toddlers acquired in advance, and a storage unit, A function to acquire data on the crying sounds of infants and toddlers, The system includes an estimation function that uses the aforementioned crying sound learning model to classify the acquired crying sound data and estimate the cause of the infant's crying, A display function that shows the aforementioned cause, A crying sound estimation program characterized in that the estimation function classifies the crying into either physiological crying or crying due to an organic physical disease, stores the crying sound data in the memory unit if the crying is classified as crying due to an organic physical disease, and the display function displays medical information.

20. A crying sound estimation method in a crying sound estimation device, comprising a crying sound learning model storage unit that stores a crying sound learning model constructed using crying sound data to which information about infants and toddlers has been previously acquired and tagged, the following: The computer of the crying sound estimation device, A crying sound data acquisition step to obtain crying sound data of infants and toddlers, An estimation step in which the crying sound data acquired is classified by speech recognition using the crying sound learning model described above, and the cause of the crying of the infant is estimated. A display step that shows the cause, Steps for suggesting solutions, A crying sound estimation method characterized in that the steps are performed, the countermeasure presentation step determines an appropriate countermeasure from a countermeasure table based on the cause of the crying of the infant estimated by the estimation step and transmits it to the display step, and if the estimation step estimates that the cause of the crying requires immediate attention, the display step displays an alert.

21. A computer equipped with a crying sound learning model storage unit that stores a crying sound learning model constructed using crying sound data that has been previously acquired and tagged with information about infants, A function to acquire data on the crying sounds of infants and toddlers, The system includes an estimation function that uses the aforementioned crying sound learning model to classify the acquired crying sound data and estimate the cause of the infant's crying, A display function that shows the aforementioned cause, A function to suggest solutions, A crying sound estimation program characterized in that, the countermeasure suggestion function determines an appropriate countermeasure from a countermeasure table based on the cause of the crying of the infant estimated by the estimation function and transmits it to the display function, and if the estimation function estimates that the cause of the crying sound requires immediate attention, the display function displays an alert.

Images