Crying sound estimation device, crying sound estimation method, and crying sound estimation program

Abstract

A crying sound estimation device, a crying sound estimation method, and a crying sound estimation program are provided that enable accurate estimation of the cause of an infant's crying, which depends on various factors. [Solution] The device comprises a microphone that captures the crying sounds of an infant, a crying sound learning model storage unit that stores a crying sound learning model, a classification unit that uses voice recognition using the crying sound learning model to classify the captured crying sounds and estimate the cause of the infant's crying, and a display unit that displays the cause, and the crying sound learning model is constructed using previously acquired crying sound data of crying sounds that have been tagged with information about the infant.

Description

[Technical Field]

[0001] The present disclosure relates to a crying sound estimation device, a crying sound estimation method, and a crying sound estimation program. [Background technology]

[0002] Prevention of postpartum depression is important, as leaving it untreated will result in enormous social losses. The main causes of childcare stress that lead to postpartum depression can be divided into four categories: infant crying, breastfeeding, isolation, and feelings of helplessness. Of these, dealing with crying is the most common and is said to have a major impact on postpartum depression.

[0003] Patent Documents 1 and 2 disclose devices that suggest the causes of infant crying and how to deal with it. The devices disclosed in Patent Documents 1 and 2 analyze the frequency of an infant's crying and estimate the infant's emotions, such as "sleepiness" or "hunger," that cause the infant to cry. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2006-84630 [Patent Document 2] Japanese Patent Application Laid-Open No. 2002-278582 Summary of the Invention [Problem to be solved by the invention]

[0005] While the cause of infant crying is often physiological crying, crying is often due to organic physical illnesses and is known to be associated with serious illnesses such as heart failure, intussusception, and meningitis. The causes and patterns of crying also change depending on the developmental stage. Thus, infant crying changes depending on many factors. The devices disclosed in Patent Documents 1 and 2 do not take these many factors into consideration.

[0006] In view of the above problems, the present invention aims to provide a crying sound estimation device, a crying sound estimation method, and a crying sound estimation program that enable accurate estimation of the cause of an infant's crying, which depends on various factors. [Means for solving the problem]

[0007] A first aspect of the present invention is a crying sound estimation device comprising: a microphone for acquiring an infant's crying sound; a crying sound learning model storage unit for storing a crying sound learning model; a classification unit for classifying the acquired crying sound by speech recognition using the crying sound learning model to estimate a cause of the infant's crying; and a display unit for displaying the cause. The crying sound learning model is constructed using crying sound data of crying sounds that have been acquired in advance and to which tags related to infant information have been added.

[0008] In the first aspect of the present invention, the classification unit may classify crying into either physiological crying or crying due to an organic physical illness.

[0009] In the first aspect of the present invention, an input unit that receives the baby's age in months may be further provided, and the classification unit may estimate the cause from the baby's age in months and the estimation result.

[0010] In a first aspect of the present invention, the system further includes an imaging device that acquires images of the infant, and an image learning model storage unit that stores an image learning model constructed using image data of images acquired in advance and tagged with information about the infant, and the classification unit classifies the acquired crying sounds using voice recognition using the crying sound learning model and image recognition using the image learning model, and may infer the cause from the image data and the classification results.

[0011] In the first aspect of the present invention, a vital data acquisition device for acquiring vital data of the infant may be further provided, and the classification section may infer the cause from the vital data and the classification result.

[0012] In the first aspect of the present invention, the vital data may be any one of the infant's heart rate, oxygen saturation level, respiratory rate, crying frequency, crying intensity, bowel sounds, respiratory sounds, and heart sounds.

[0013] In the first aspect of the present invention, the vital data acquisition device may be a sound collection sensor placed in contact with the body of the infant.

[0014] In the first aspect of the present invention, a language input unit that accepts input of a language normally used by a caregiver of an infant may be further provided, and the classification unit may infer the cause from the language and the classification result.

[0015] In the first aspect of the present invention, the device may further include a solution presentation unit, which may determine an appropriate solution from a solution table based on the cause of the infant's crying estimated by the classification unit and send it to the display unit.

[0016] In the first aspect of the present invention, a memory unit is further provided, and when the classification unit classifies the crying as crying caused by an organic physical disease, the crying sound data may be stored in the memory unit and medical information may be displayed on the display unit.

[0017] In the first aspect of the present invention, when the classification unit estimates that the cause of the crying sound requires urgent attention, the solution presentation unit may cause the display unit to display an alert.

[0018] In the first aspect of the present invention, the device may further include a memory unit, and after the classification unit estimates the cause of the infant's crying, the memory unit may store the infant's crying sound data, information identifying the infant, the date and time the crying sound was acquired, the duration of the crying, and the estimated cause.

[0019] A second aspect of the present invention is a crying sound estimation method, which is a crying sound estimation device having a crying sound learning model storage unit that stores a crying sound learning model constructed using crying sound data tagged with previously acquired information about an infant, and includes an acquisition step of acquiring crying sound data from an infant, an estimation step of classifying the acquired crying sound data by voice recognition using the crying sound learning model to estimate the cause of the infant's crying, and a display step of displaying the cause.

[0020] A third aspect of the present invention is a crying sound estimation program, which is implemented by a computer having a crying sound learning model storage unit that stores a crying sound learning model constructed using previously acquired crying sound data tagged with information about the infant, and which has the following functions: an acquisition function that acquires crying sound data from an infant; an estimation function that classifies the acquired crying sound data by voice recognition using the crying sound learning model to estimate the cause of the infant's crying; and a display function that displays the cause.

[0021] According to the present invention, it is possible to provide a crying sound estimation device, a crying sound estimation method, and a crying sound estimation program that enable accurate estimation of the cause of an infant's crying, which depends on various factors. [Brief explanation of the drawings]

[0022] [Figure 1] 1 is a schematic diagram illustrating an example of a crying sound estimation device according to a first embodiment. [Figure 2] 1 is a block diagram showing an example of the configuration and functional units of a crying sound estimation device according to a first embodiment. [Figure 3] FIG. 2 is a schematic diagram showing how a crying sound estimation device according to the first embodiment displays, on a display unit, a method of dealing with an estimated cause of crying. [Figure 4] FIG. 4 is a schematic diagram showing how the crying sound estimation device according to the first embodiment displays, on a display unit, a next solution to the estimated cause of crying. [Figure 5] FIG. 10 is a schematic diagram illustrating an example of a sound pickup sensor of a crying sound estimation device according to a second embodiment. [Figure 6] FIG. 10 is a schematic diagram showing another example of a sound pickup sensor of the crying sound estimation device according to the second embodiment. [Figure 7] 1 is a flowchart illustrating a crying sound estimation method according to the first embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0023] Next, an embodiment of the present invention will be described with reference to the drawings. In the description of the drawings relating to the embodiment, the same or similar parts are designated by the same or similar reference numerals. However, it should be noted that the drawings are schematic, and the relationship between planar dimensions and the like may differ from the actual ones. Therefore, specific dimensions should be determined with reference to the following description. Furthermore, it goes without saying that the drawings may include parts with different dimensional relationships and ratios.

[0024] Furthermore, the embodiments are merely examples of devices and methods for embodying the technical idea of ​​the present invention, and the technical idea of ​​the present invention does not limit the configuration, arrangement, layout, etc. of each component to those described below. The technical idea of ​​the present invention can be modified in various ways within the technical scope defined by the claims.

[0025] (First embodiment) The crying sound estimation device according to this embodiment is a device that acquires the crying sound of a crying infant 1 by a microphone, estimates the cause of the crying based on the acquired crying sound, and presents the cause, as shown in the example of Fig. 1. In the example shown in Fig. 1, the crying sound of the infant 1 is acquired by a microphone built into a mobile terminal 2, and the cause estimated by the crying sound estimation device is displayed on a display 3 of the mobile terminal 2.

[0026] An example of the configuration and functional units of the crying sound estimation device 10 according to this embodiment is shown in Fig. 2. The crying sound estimation device 10 shown in Fig. 2 includes a CPU 201 for executing various calculations, a ROM 202 for storing processing programs, a RAM 203 for storing data and the like, a storage unit 204 for storing various data and calculation results and the like, an I / O (input / output interface) 205, a microphone 206, a display unit 207, an input unit 209, and the like.

[0027] The I / O 205 is an interface, buffer, etc. for communication (transmission and reception).

[0028] The microphone 206 picks up the sound of the baby crying.

[0029] The crying sound estimation device 10 according to this embodiment may also be connected to an input keyboard, mouse, etc.

[0030] The crying sound estimation device 10 is various electronic computers (computational resources) such as a mobile terminal, a personal computer (PC), a mainframe, a workstation, a cloud computing system, etc. In the example shown in Fig. 1 , the crying sound estimation device 10 is a mobile terminal, the microphone 206 is built into the mobile terminal, and the display unit 207 is a display of the mobile terminal. The microphone 206 of the crying sound estimation device 10 according to this embodiment is not limited to the example shown in Fig. 1 , and may be, for example, a microphone externally connected to a personal computer or the like, or may be a form in which crying sound data acquired in advance by a microphone not connected to a personal computer or the like is input to a personal computer or the like.

[0031] 2 also shows functional units within the CPU 201. When each functional unit of the CPU 201 is realized by software, the CPU 201 realizes it by executing instructions of a program, which is software that realizes each function. In detail, the CPU 201 includes a classification unit 208 and the like. The storage unit 204 includes a crying sound learning model storage unit 210.

[0032] The crying sound learning model storage unit 210 stores a crying sound learning model. The crying sound learning model is constructed using crying sound data of crying sounds to which tags related to information about the infant are attached, which have been acquired in advance.

[0033] The classification unit 208 classifies the acquired crying sounds by voice recognition using a crying sound learning model, and estimates the cause of the baby's crying.

[0034] The display unit 207 displays the cause of the infant's crying estimated by the classification unit 208. In the example shown in Fig. 1, as an example, a circle, triangle, or other line is displayed on the display 3 to indicate that the crying sound is being acquired, and causes estimated by the classification unit 208 are displayed such as "hungry," "hold me," "sleepy," and "hot." Furthermore, as will be described later with reference to Figs. 3 and 4, a solution according to the cause of the crying may be displayed.

[0035] The input unit 209 may accept input of the infant's date of birth. The input unit 209 may also accept input of the infant's age in months, developmental stage, language used by the caregiver, gender, height, weight, etc. Furthermore, as will be described later, the input unit 209 may accept a determination result by the caregiver as to whether the cause of the crying estimated by the classification unit 208 was correct. The input information about the infant may be stored as infant information in the storage unit 204. When classifying the acquired crying sounds and estimating the cause of the infant's crying, the classification unit 208 may read out the infant information stored in the storage unit 204 and use the infant information when classifying the crying sounds and estimating the cause.

[0036] The crying sound estimation device 10 according to this embodiment will be described below with reference to the example shown in Fig. 1. As shown in Fig. 1, when an infant 1 cries, the crying sound of the infant 1 is recorded by the microphone 206 of the crying sound estimation device 10. The microphone 206 records the crying sound and generates crying sound data. The microphone 206 transmits the generated crying sound data to the classification unit 208.

[0037] The classification unit 208 classifies the acquired crying sounds by speech recognition using a crying sound learning model. The crying sound learning model is constructed using crying sound data of crying sounds tagged with information about the infant, which has been acquired in advance, and is stored in the crying sound learning model storage unit 210.

[0038] The causes of crying in infants can be classified into physiological causes and those caused by organic physical illness. Physiological crying accounts for approximately 95% of crying and is a response to hunger, discomfort (e.g., a wet diaper), or separation. Physiological crying stops when the infant's needs are met through measures such as feeding, changing a diaper, or holding.

[0039] Crying due to organic physical illness accounts for approximately 5% of all crying. Generally, illnesses in which some kind of abnormality has occurred in the organs themselves, such as inflammation or cancer, are called organic physical illnesses, while other illnesses, including those with no identified cause, are called functional physical illnesses. For this reason, crying due to illnesses with an identified cause is called crying due to organic physical illness.

[0040] The training data for constructing the crying sound learning model was generated by recording infant cries in advance, converting the recorded crying sound data into spectrogram format, and then tagging the cause of the crying. The crying sound learning model was constructed by performing deep learning using the generated training data.

[0041] If the infant's crying was physiological crying, the cause of the crying tagged to the crying sound when generating training data was the method of dealing with the situation when the infant stopped crying.

[0042] If an infant's crying is due to an organic physical illness, the infant will not stop crying even if measures taken to treat physiological crying are taken. Therefore, if the crying is presumed to be due to an organic physical illness, such as when measures taken assuming that the crying is physiological crying do not stop the infant, the crying should be recorded and the infant should be examined by a pediatrician or other doctor. If a system that allows online pediatrician consultations has been established in advance, it is possible to record the crying during the online consultation. Training data was generated by tagging the recorded crying data with the doctor's diagnosis and medical record information.

[0043] In the crying sound estimation device 10 according to this embodiment, a crying sound learning model may be constructed by performing deep learning using training data generated when the infant's crying is physiological crying and training data generated when the infant's crying is due to an organic physical disease. By using the learning model constructed in this manner, the classification unit 208 can classify the acquired crying sound by voice recognition into either physiological crying or crying due to an organic physical disease.

[0044] When the classification unit 208 classifies the cry of the acquired crying sound as a physiological cry, the classification unit 208 can further estimate the cause of the crying of the acquired crying sound.

[0045] The number of emotion categories for infants changes with age. Immediately after birth, there are only two categories, pleasant and unpleasant, but by the time the infant is about one year old, this number increases to 18 categories, including affection and fear. Conventional devices for analyzing infant cries classify infant emotions into approximately five categories and select the cause of the crying from one of the five classified emotions. The cry sound estimation device 10 according to the present embodiment classifies infant emotions into 18 categories according to age and selects the cause of the crying from one of the 18 classified emotions. The cry sound estimation device 10 according to the present embodiment may select the cause of the crying from one of the 18 emotions and then display the selection result on the display unit 207. The cry sound estimation device 10 according to the present embodiment may display the emotion on the display unit 207 as the selection result. However, if the cause of the crying has been estimated, it is preferable that the cry sound estimation device 10 according to the present embodiment display the cause of the crying on the display unit 207 as the selection result. For example, when the emotion selected as the cause of the crying is discomfort, if the cause of the discomfort is hunger, the crying sound estimation device 10 according to this embodiment may display "hunger" on the display unit 207.

[0046] Table 1 shows the types of emotions classified by the crying sound estimation device 10 according to this embodiment according to the age in months. The emotions shown in Table 1 are the emotions of an infant added at the corresponding age in months, and typically, the emotions of an infant are added starting from those shown in the top row of Table 1. For example, an infant aged 0 to 1 month first has the emotions of "pleasure," "discomfort," and "interest," and then an infant aged 1 to 2 months has the emotions of "anxiety," "relief," and "loneliness" in addition to the emotions of "pleasure," "discomfort," and "interest."

[0047] [Table 1]

[0048] If the infant's crying was physiological crying, the caregiver's response when the infant stopped crying was considered the cause of the crying. The crying sound was recorded for each infant cry, and at the same time the caregiver selected the infant's emotion from the emotions shown in Table 1 according to the infant's age or developmental stage. The recorded crying sound was tagged with the emotion selected by the caregiver according to the age or developmental stage, and training data was generated.

[0049] In the crying sound estimation device 10 according to this embodiment, a crying sound learning model may be constructed by performing deep learning using training data generated by classifying the causes of an infant's crying into emotions according to the infant's age in months. By using the learning model constructed in this manner, the classification unit 208 can classify the causes of the acquired crying sounds into 18 types of emotions through voice recognition using the infant's age in months input by the input unit 209.

[0050] When an infant's crying is due to an organic physical illness, the primary causes are classified as cardiac, gastrointestinal, infectious, and traumatic. Among these causes, potentially life-threatening conditions include heart failure, intussusception, intestinal volvulus, meningitis, and intracranial hemorrhage due to head trauma. Examples of potentially urgent conditions that can be predicted from the cry include the whistling sound of an asthma attack, which is caused by airway narrowing. Meningitis cries are characterized by high-pitched, rhythmically unstable infant cries due to increased intracranial pressure or severe headaches. Asphyxiation cries have a higher-than-normal fundamental frequency (f0) due to abnormal vocal cord vibration caused by respiratory distress. The crying sound caused by intussusception becomes a sudden, intense crying sound, and the intensity and frequency of the crying sound fluctuate periodically as the intestines contract and relax. By capturing these characteristics, it is possible to detect the disease. When the classification unit 208 classifies the acquired crying sound as crying caused by an organic physical disease, the classification unit 208 estimates the cause of the acquired crying sound. The crying sound estimation device 10 may further suggest, for example, visiting a pediatrician or obtaining vital sign data, as described below.

[0051] As a specific example of a disease that can be estimated by the crying sound estimation device 10 according to this embodiment, sudden infant death syndrome (SIDS) will be described. SIDS mainly occurs during nighttime sleep, most commonly occurring between 2 and 6 months of age. While the cause of the syndrome is unknown, it is thought to involve a decreased awakening response when breathing is suppressed for some reason. SIDS accounts for 6.1% of infant deaths, and its incidence in Japan is estimated to be approximately 1 in 6,000 to 7,000 births. It has been revealed in papers ("The Cry Characteristics of an Infant Who Died of the Sudden Infant Death Syndrome" (RH Colton, 1981), "Cry features in siblings of SIDS" (Michael P Robb, 2007), and "Sudden Infant Death Syndrome: Cry characteristics" (Michael P Robb, 2013)) that when 14 cries from four-day-old infants who died suddenly of unknown causes were compared with a control group of newborns in terms of nine acoustic characteristics, including fundamental frequency (f0), duration, formant frequency, and sound pressure level, the cries had lower f0, longer duration, lower formant frequency, and higher sound pressure level across the entire range. Using these results, the cry sound estimation device 10 according to this embodiment can estimate the risk of sudden infant death syndrome from the sounds of infant cries.

[0052] Furthermore, when estimating the risk of sudden infant death syndrome, the accuracy can be improved by using oxygen saturation data. As will be described later, the crying sound estimation device 10 according to this embodiment may acquire vital data of the infant, particularly oxygen saturation data, and estimate the cause of the crying using the oxygen saturation data in addition to the crying sound data.

[0053] As described above, the classification of crying into physiological crying or crying due to an organic physical illness can be performed immediately after the crying data is acquired, but there are illnesses that become apparent only when crying sounds are continuously acquired. After classifying the crying based on the acquired crying sound data, the classification unit 208 may store the infant crying sound data, for example, in the memory unit 204 as a crying sound database together with information identifying the infant, the date and time the crying sound was acquired, the duration of the crying, and the classification results by the classification unit 208. The classification unit 208 stores the above data in the crying sound database each time an infant crying sound is acquired, thereby accumulating crying sound data.

[0054] For example, colic refers to excessive crying that occurs in infants aged four months or younger, that has no identifiable organic cause, and that continues for three hours or more per day, three days or more per week for three weeks or more. The classification unit 208 classifies the acquired crying sounds into physiological crying or organic crying by voice recognition using a crying sound learning model, and can also deduce diseases such as colic that are revealed by continuously acquired crying sound information by comparing the acquired crying sound with crying sound data stored in a crying sound database.

[0055] As described above, abnormal crying of an infant can be detected by estimating the crying sound by using voice recognition of the crying sound in the crying sound estimation device 10. However, when detecting abnormal crying sounds, the crying sound estimation device 10 according to the present embodiment particularly utilizes a technology called TS-VAD (Target Speaker Voice Activity Detection). TS-VAD learns the crying sounds to be detected in advance and detects the timing and duration of the crying sounds that are detected among the acquired crying sounds. The crying sound estimation device 10 according to the present embodiment uses this technique to detect abnormal crying sounds and can determine, from the abnormal crying sounds, whether the underlying illness requires emergency treatment and transportation by ambulance or other means, or whether the crying is not urgent and a medical examination is sufficient.

[0056] Furthermore, the cry sound estimation device 10 according to this embodiment can also detect abnormal crying by detecting that "the crying is different from normal crying." Even if the cry sound estimation device 10 does not detect any abnormality, such as a disease, through voice recognition of the crying sound, the crying may be different from normal due to a disease or other cause. Therefore, detecting abnormal crying enables early intervention and treatment of diseases, etc. Furthermore, information that the crying is different from normal can help doctors make an early diagnosis. Furthermore, accumulating analysis data of crying over a long period of time can more accurately detect abnormalities in infants and identify the causes.

[0057] The classification unit 208 may detect abnormal crying by comparing the acquired crying sound with crying sound data stored in a crying sound database in addition to classifying the acquired crying sound into physiological crying or organic crying by voice recognition using a crying sound learning model. For example, the classification unit 208 may classify the acquired crying sound data into physiological crying or organic crying, and then detect whether the crying is abnormal by comparing the acquired crying sound data with crying sound data stored in the crying sound database that has been classified as the same as or similar to the classification result by the classification unit 208.

[0058] The cry sound estimation device 10 according to this embodiment can visualize health data of an infant by storing the infant's cries in a cry sound database. For example, the cry sound estimation device 10 can identify the infant's developmental stage by comparing the changes over time in the causes of crying classified as physiological crying stored in the infant cry sound database with the types of emotions corresponding to the infant's age in months shown in Table 1, and display the identified developmental stage on the display unit 207.

[0059] The types of emotions corresponding to the infant's age in months shown in Table 1 represent average emotions for infants. However, if there is a delay of a certain level or more in the infant's developmental stage identified from the data stored in the cry sound database compared to the infant's actual age in months, language development delay or developmental disorder is suspected. The cry sound estimation device 10 can detect data in which delays and abnormalities in emotional differentiation are observed. The cry sound estimation device 10 may conduct a medical interview for detected data in which delays or abnormalities are observed, for example, by displaying a medical questionnaire on the display unit 207, and further, enable early rehabilitation by displaying information about medical institutions on the display unit 207.

[0060] Respiratory syncytial virus (RSV) infection can be fatal, resulting in sudden death due to apnea, respiratory failure due to acute bronchiolitis, and acute myocarditis. After infection, RSV first causes upper respiratory tract inflammation, followed by symptoms such as runny nose and cough. Approximately 30% to 40% of cases progress to acute bronchiolitis and pneumonia, accompanied by symptoms such as tachypnea and wheezing. RSV is likely to spread to the lower respiratory tract in infants, and infants under six months of age are particularly susceptible to severe illness. Furthermore, premature infants and infants with underlying conditions such as congenital heart disease are at even higher risk of developing severe illness. Because a correlation is believed to exist between crying sounds and the risk of developing severe RSV infection, early detection and prevention of severe illness can be achieved by detecting the risk of severe illness in infants using the cry sound estimation device 10 and vital signs according to this embodiment.

[0061] The sounds of an infant's cry vary depending on the language that the infant normally hears, i.e., the language normally used by the caregiver, such as the caregiver's native language. Therefore, in the process of generating the training data generated according to the cause of crying and the baby's age as described above, the language normally used by the caregiver may be added as a tag to generate the training data. In the crying sound estimation device 10 according to this embodiment, a crying sound learning model may be constructed by performing deep learning using the training data generated in this manner. The classification unit 208 uses the language primarily used by the caregiver of the infant, whose input is received by the input unit 209, to perform speech recognition using this crying sound learning model, thereby enabling more accurate classification when classifying the cause of crying from the acquired crying sounds.

[0062] The crying sound estimation device 10 according to this embodiment may include an imaging device for capturing an image of an infant. If the crying sound estimation device 10 is, for example, a mobile terminal as shown in Fig. 1, the imaging device may be a video camera included in the mobile terminal, but is not limited to this and may be, for example, an externally connected camera.

[0063] When capturing the sound of an infant's cry, an image capturing device may be used to capture the infant's crying and obtain image data of the infant. For example, there is a correlation between facial movements and emotions, as shown in Table 2. By using not only the sound of an infant's cry but also the facial color, facial expression, and limb movements during crying to infer the cause of the crying, the accuracy of the cause inference can be improved.

[0064] [Table 2]

[0065] When classifying the acquired crying sounds, the classification unit 208 may perform classification by voice recognition using a crying sound learning model for the crying sound data of the infant and image recognition using an image learning model for the image data of the infant. The crying sound estimation device 10 according to this embodiment may further include an image learning model storage unit. The image learning model is constructed using image data of a crying infant that has been acquired in advance and to which tags related to infant information have been added, and is stored in the image learning model storage unit.

[0066] The information about the infant that is added to the image data when constructing the image learning model includes the cause of the crying, and may also include at least one of the infant's age in months, developmental stage, and the caregiver's primary language.

[0067] The cry sound estimation device 10 according to this embodiment may acquire vital data of an infant and estimate the cause of the crying using the vital data in addition to the cry sound data. Examples of the infant's vital data include the infant's heart rate, oxygen saturation level, respiratory rate, crying frequency, crying intensity, intestinal peristalsis sounds, breath sounds, and heart sounds. The infant's vital data is acquired by a vital data acquisition device. The vital data acquisition device is a general device for acquiring the infant's heart rate, oxygen saturation level, respiratory rate, crying frequency, crying intensity, intestinal peristalsis sounds, breath sounds, and heart sounds, which are mentioned above as examples of infant vital data, and may be connected to the cry sound estimation device 10. Alternatively, the vital data of an infant may be acquired and only the acquired vital data may be input into the cry sound estimation device 10. Since frequent burping is actually correlated with illness, frequency is also added. (→ Commonly seen in gastroesophageal reflux disease in newborns and tic disorders in young children)

[0068] Figures 5 and 6 show examples of sound collection sensors for acquiring vital data of infants. Figure 5 shows a state in which sound collection sensor 53 is attached to the belly side of diaper 52 of infant 51, and Figure 6 shows a state in which sound collection sensor 63 is attached to the back side of diaper 62 of infant 61.

[0069] The sound collection sensors 53, 63 shown in Figures 5 and 6 are attached to the diaper with tape, clips, etc., are detachable, and can be attached to any position on the diaper. The sound collection sensors 53, 63 shown in Figures 5 and 6 are not limited to being attached to diapers, but may also be attached to clothing, etc. The sound collection sensors 53, 63 are connected via wireless communication to a crying sound estimation device 10 (not shown), and data acquired by the sound collection sensors 53, 63 is transmitted to the crying sound estimation device 10 via wireless communication. The sound collection sensors 53, 63 are placed in contact with the body of an infant to acquire heart sounds, respiratory sounds, and intestinal peristalsis sounds, and by combining these with symptoms such as diarrhea, vomiting, and difficulty breathing, diseases such as choking and intestinal obstruction can be predicted.

[0070] Obtaining vital data is not as easy as recording the sound of crying or taking images, for example, because it requires a device to be attached to the baby's body. Therefore, this may be a form that is used as needed when the baby's crying is organic crying, and illness or physical abnormality of the baby is suspected.

[0071] The crying sound estimation device 10 according to this embodiment may further include a remedy presentation unit and a remedy table storage unit that stores a remedy table listing the relationship between the causes of an infant's crying and the remedy. The remedy presentation unit may determine an appropriate remedy from the remedy table based on the cause of the infant's crying estimated by the estimation unit, transmit the determined remedy to the display unit 207, and display the determined remedy on the display unit 207, thereby presenting the remedy.

[0072] If the infant's crying is physiological crying, the solution suggestion unit displays solutions according to the cause of the crying on the display unit 207, such as holding the infant, breastfeeding, patting the head or body, rocking the body by using a bouncer or holding the infant, putting a pacifier in the baby's mouth, breastfeeding, etc. For example, as shown in Fig. 3, if the cause of the crying is hunger, the display unit 207 displays "give milk" as a solution. If the crying does not stop, the input unit may accept input from the caregiver or the like that the crying does not stop even after trying the solutions suggested by the solution suggestion unit, and the solution suggestion unit may suggest a solution that is estimated to be the next most effective after the solution suggested previously. As shown in FIG. 4, if the crying does not stop even when hunger is assumed to be the cause of the crying and the solution "give milk" is shown on the display unit 207, the cause of the crying may be assumed to be fatigue rather than hunger, and the next solution may be presented.

[0073] If the infant's crying is due to an organic physical illness, or if the infant's crying is physiological but the crying does not stop despite any of the remedies suggested by the remedies suggestion unit, the remedies suggestion unit suggests remedies such as connecting to an online consultation depending on the cause of the crying, including information on local commercial medical care systems, childcare and pediatric consultation centers, consultation hours and contact details of medical institutions, etc., and if a system has been established that allows online consultations with a pediatrician.

[0074] Furthermore, the solution suggestion unit may suggest acquiring vital data of the infant if the infant's crying is due to an organic physical illness, or if the infant's crying is physiological crying but the crying does not stop despite any of the solutions suggested by the solution suggestion unit. Furthermore, for example, if a vital data acquisition device is connected to the cry sound estimation device 10 and is in a state where vital data can be acquired, the solution suggestion unit may prompt the cry sound estimation device 10 to acquire vital data of the infant, for example, by sending a signal to the vital data acquisition device to start measurement.

[0075] If the classification result performed by the classification unit 208 of the crying sound estimation device 10 of this embodiment indicates that the cause of the crying sound requires emergency response, the countermeasure presentation unit may display an alert on the display unit 207, and may also display information about medical institutions in case of emergency on the display unit 207 together with the alert.

[0076] The crying sound estimation method according to this embodiment will be described with reference to the flowchart in Fig. 7. In Fig. 7, the flow is assumed to be performed in a crying sound estimation device including a crying sound learning model storage unit that stores a crying sound learning model constructed using crying sound data that has been acquired in advance and to which tags related to information about the infant have been added.

[0077] In step S701, crying sound data of an infant is acquired (acquisition step).

[0078] In step S702, the acquired crying sound data is classified by voice recognition using a crying sound learning model, and the cause of the baby's crying is estimated (estimation step).

[0079] In step S703, the cause is displayed (display step).

[0080] As mentioned above, the present invention naturally includes various embodiments not described herein. Therefore, the technical scope of the present invention is defined only by the invention-specifying matters according to the scope of the claims that are appropriate from the above description. [Explanation of symbols]

[0081] 1. Infants and young children 2. Mobile devices 3. Display 10 Crying sound estimation device 201 CPU 202 ROM 203 RAM 204 Storage section 205 I / O (input / output interface) 206 Microphone 207 Display section 208 Classification Department 209 Input section 210 Crying sound learning model memory section 51, 61 Infants 52, 62 Diapers 53, 63 Sound collection sensor

Claims

1. A microphone that captures the crying sounds of infants and young children; a crying sound learning model storage unit that stores a crying sound learning model; a classification unit that classifies the acquired crying sounds by speech recognition using the crying sound learning model and estimates the cause of the crying of the infant; a display unit that displays the cause; The crying sound estimating device is characterized in that the crying sound learning model is constructed using crying sound data of crying sounds that have been acquired in advance and to which tags related to information about infants and young children have been added.

2. The crying sound estimating device according to claim 1 , wherein the classifying unit classifies the crying into either physiological crying or crying due to an organic physical illness.

3. The crying sound estimation device according to claim 1 , further comprising an input unit that receives the baby's age in months, wherein the classification unit estimates the cause from the baby's age in months and the result of the estimation.

4. an imaging device for capturing an image of the infant; an image learning model storage unit that stores an image learning model constructed using image data of images that have been previously acquired and to which tags related to information about infants and young children have been added; the classification unit classifies the acquired crying sound by speech recognition using the crying sound learning model and image recognition using the image learning model, and estimates the cause from the image data and the estimation result.

5. The crying sound estimation device according to claim 1 , further comprising a vital data acquisition device that acquires vital data of the infant, wherein the classification unit estimates the cause from the vital data and the result of the estimation.

6. 6. The crying sound estimating device according to claim 5, wherein the vital data is at least one of the infant's heart rate, oxygen saturation level, respiratory rate, crying frequency, crying intensity, intestinal peristalsis sounds, respiratory sounds, and heart sounds.

7. 6. The crying sound estimating device according to claim 5, wherein the vital data acquiring device is a sound collecting sensor installed so as to come into contact with the body of the infant.

8. 2. The crying sound estimation device according to claim 1, further comprising a language input unit that accepts input of a language normally used by a caregiver of the infant, and wherein the classification unit estimates the cause from the language and the result of the estimation.

9. 2. The crying sound estimation device according to claim 1, further comprising a solution presentation unit, wherein the solution presentation unit determines an appropriate solution from a solution table based on the cause of the infant's crying estimated by the classification unit, and transmits the solution to a display unit.

10. The crying sound estimation device according to claim 2, further comprising a storage unit, wherein when the classification unit classifies the crying as crying caused by an organic physical disease, the crying sound data is stored in the storage unit and medical information is displayed on the display unit.

11. The crying sound estimation device according to claim 9, wherein when the classification unit estimates that the cause of the crying sound requires emergency response, the solution presentation unit causes the display unit to display an alert.

12. 2. The crying sound estimation device according to claim 1, further comprising a storage unit, wherein after the classification unit estimates the cause of the infant's crying, the storage unit stores the crying sound data of the infant, information identifying the infant, the date and time the crying sound was acquired, the duration of the crying, and the estimated cause.

13. A crying sound estimation device including a crying sound learning model storage unit that stores a crying sound learning model constructed using crying sound data that has been previously acquired and to which tags related to information about an infant are added, an acquisition step of acquiring crying sound data of an infant; an estimation step of classifying the acquired crying sound data by voice recognition using the crying sound learning model and estimating the cause of the crying of the infant; a display step of displaying the cause; A crying sound estimation method comprising:

14. A computer includes a cry sound learning model storage unit that stores a cry sound learning model constructed using previously acquired cry sound data to which tags related to information about infants and young children are attached. An acquisition function to acquire crying sound data of infants and young children, an estimation function for classifying the acquired crying sound data by voice recognition using the crying sound learning model and estimating the cause of the crying of the infant; a display function for displaying the cause; A crying sound estimation program characterized by realizing the above.

Images