system

Abstract

We provide the system. [Solution] A measuring device for acquiring biological data, Sensor devices for acquiring environmental data, A voice acquisition device for acquiring voice data, An analytical device that integrates and analyzes the acquired data and evaluates the state of the target, A generating device that generates recommended countermeasures based on the aforementioned analysis results, A notification device that notifies the generated countermeasures, A system that includes this.

Description

Technical Field

[0005]

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor and includes steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In modern society, many parents experience excessive stress and lack of sleep due to insufficient knowledge about child-rearing and not knowing how to deal with a baby's night crying. In particular, in order to deal with night crying whose cause is unclear, it is necessary to comprehensively consider the crying sound, environmental changes, and the baby's biological information, but there is no efficient support system to achieve this. In order to solve such problems, there is an increasing demand for a system that can comprehensively analyze crying sounds, environmental data, and biological data and provide specific solutions.

Means for Solving the Problems

[0005] To solve this problem, the present invention provides a measuring device for acquiring biological data, a sensor device for acquiring environmental data, and a voice collection device for acquiring voice data, and further provides an analysis device for integrating and analyzing the acquired data. The analysis device evaluates the condition of the subject, and based on the results, a generation device generates recommended countermeasures. The generated countermeasures are notified to the parent through a notification device, enabling a quick response. In addition, the analysis device has an automatic learning function and can improve the accuracy of recommendations by receiving feedback from the user. In this way, it provides personalized support that is tailored to the individual childcare situation.

[0006] A "measuring device" is a device used to acquire a baby's biological information, and for example, it has the function of measuring body temperature, heart rate, and movement.

[0007] A "sensor device" is a device that collects information about the surrounding environment, and for example, has functions to detect temperature, humidity, and lighting levels.

[0008] A "sound collection device" is a device used to acquire sound data, such as a baby crying, and has the function of converting sound into digital data using a microphone or similar device.

[0009] An "analysis device" is a device that integrates acquired biological data, environmental data, and audio data, performs data analysis, and executes algorithms to evaluate the state of the subject.

[0010] A "generating device" is a device that has the function of generating recommended countermeasures based on the evaluation results from an analytical device.

[0011] A "notification device" is a device that communicates generated countermeasures to parents, has a notification function, and enables real-time intervention.

[0012] The "automatic learning function" is a feature that allows the analysis device to improve its analysis algorithm based on user feedback, thereby enhancing the accuracy of its recommendations. [Brief explanation of the drawing]

[0013] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2, which incorporates an emotion engine. [Figure 14]It is a sequence diagram showing the processing flow of a data processing system in Application Example 2 when a sentiment engine is combined.

Embodiments for Carrying Out the Invention

[0014] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0015] First, the terms used in the following description will be explained.

[0016] In the following embodiments, a numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0017] In the following embodiments, a numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0018] In the following embodiments, a numbered storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, etc.

[0019] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0020] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0021] [First Embodiment]

[0022] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0023] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0024] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0025] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0026] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0027] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0028] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0029] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0030] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0031] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0032] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0033] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0034] This invention is a support system for solving the problem of nighttime crying in infants during childcare, and includes a measuring device, a sensor device, a sound collection device, an analysis device, a generation device, and a notification device. This system has the function of collecting and integrating the infant's biological data, surrounding environment data, and crying sound data, and analyzing this data to identify the cause of nighttime crying and notify parents of appropriate countermeasures.

[0035] First, the device uses a wearable measuring device to acquire information about the baby's body temperature, heart rate, and movement. This information is usually acquired in real time and used to monitor the baby's health and stability. The device also acquires environmental information such as temperature and humidity using sensors installed in the room. This allows the device to constantly know whether the room is a comfortable environment for the baby.

[0036] The terminal's voice collection device records the baby's cries and generates audio data. This audio data is an important source of information for analyzing crying patterns. The obtained crying data is analyzed by a voice recognition algorithm, and the psychological and physiological state corresponding to the crying pattern is inferred.

[0037] Next, the server integrates this data and uses an analysis device to make a comprehensive assessment of the baby's condition. For example, even if the crying pattern indicates "hunger," it compares this data with other information such as body temperature and ambient temperature to determine the most appropriate course of action.

[0038] Based on the analysis results, the generator produces specific countermeasures. These countermeasures are presented in an intuitive and easy-to-understand way for parents, such as, "It is highly likely that the child is hungry, so we recommend breastfeeding."

[0039] The generated countermeasures are notified to the parent from the device via a notification device. The user can then take appropriate action immediately based on this notification. Furthermore, the parent's response is input into the system as feedback, which the server analyzes and uses its automatic learning function to improve the accuracy of future recommendations. This enables the continuous provision of personalized support.

[0040] The introduction of this system will allow parents to deal with their baby's nighttime crying problem with less stress and less time, and as a result, a significant improvement in the childcare environment is expected.

[0041] The following describes the processing flow.

[0042] Step 1:

[0043] The device uses a wearable measuring device to acquire biometric data such as the baby's body temperature, heart rate, and movement in real time.

[0044] Step 2:

[0045] The terminal uses sensor devices installed in the room to collect environmental data such as temperature, humidity, and lighting intensity.

[0046] Step 3:

[0047] The device uses a voice collection device to record the baby's cries and generate audio data.

[0048] Step 4:

[0049] The terminal sends all the data acquired in steps 1, 2, and 3 to the server via the communication module.

[0050] Step 5:

[0051] The server analyzes the received audio data, uses a speech recognition algorithm to identify crying patterns, and infers the state (for example, finding signs of hunger or discomfort).

[0052] Step 6:

[0053] The server integrates biometric and environmental data, and performs data analysis to assess the baby's overall condition, along with the results of the crying analysis.

[0054] Step 7:

[0055] Based on the analysis results, the server generates the optimal countermeasures for the identified causes via a generator.

[0056] Step 8:

[0057] The server sends the generated corrective actions to the terminal.

[0058] Step 9:

[0059] The device notifies the parent and provides recommended actions in real time (e.g., "The child may be hungry. Breastfeeding is recommended").

[0060] Step 10:

[0061] The user responds to the baby's needs based on the notification and inputs the results and feedback into the system via their device.

[0062] Step 11:

[0063] The server analyzes user feedback, updates its data analysis model using an automated learning algorithm, and aims to improve the accuracy of future countermeasure suggestions.

[0064] (Example 1)

[0065] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0066] A baby's nighttime crying is a major source of stress and burden for parents. Traditional methods often make it difficult to identify and address the cause of a baby's crying, and finding an appropriate solution is challenging. Therefore, there is a need for a system that can accurately identify the cause of a baby's crying and quickly implement appropriate countermeasures.

[0067] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0068] In this invention, the server includes measuring means for acquiring biological information, detection means for acquiring environmental information, and sound collection means for acquiring sound information. This enables comprehensive data analysis of the baby's physiological state and surrounding environment, allowing for the identification of the cause of the baby's crying and the rapid presentation of appropriate countermeasures to the parents.

[0069] "Biometric information" refers to data related to a baby's physical condition and reactions, such as body temperature, heart rate, and movement.

[0070] "Environmental information" refers to data related to the surrounding environmental conditions, such as temperature, humidity, and sound pressure in the space where the baby is located.

[0071] "Auditory information" refers to information expressed through sound, such as a baby crying.

[0072] "Measuring means" refers to devices and methods for acquiring biological information, which allows for monitoring the baby's health.

[0073] "Detection means" refers to devices and methods for sensing and acquiring environmental information, which allows us to obtain information to determine the comfort level of the environment in which a baby is present.

[0074] "Voice collection means" refers to devices and methods for acquiring voice information, which can be used to obtain data for analyzing patterns in a baby's cries.

[0075] "Analysis means" refers to devices and methods for integrating and analyzing acquired biological information, environmental information, and audio information to evaluate the baby's condition.

[0076] "Generating means" refers to devices or methods that create recommended countermeasures based on analyzed data.

[0077] "Notification means" refers to devices or methods for informing users, such as parents, of the generated countermeasures, thereby enabling a quick response.

[0078] This invention is a support system for resolving the problem of babies crying at night. This system consists of a server, terminals, and users, and acquires biometric information, environmental information, and voice information. By integrating and analyzing this data, it evaluates the baby's condition, generates appropriate countermeasures, and notifies the user.

[0079] The device acquires real-time biometric information such as the baby's body temperature, heart rate, and movement through a wearable measuring device. It also collects environmental information such as room temperature and humidity using environmental sensors. This allows for constant monitoring of whether the baby's living space is comfortable. Furthermore, a voice recording device records the baby's cries, acquiring audio information. All of this data is transmitted to a server via the network.

[0080] The server integrates the received data and uses advanced analytical equipment to comprehensively evaluate the baby's psychological and physiological state. This analysis utilizes a generative AI model, for example, analyzing the characteristics of crying to infer causes such as "hunger" or "discomfort." Based on the analysis results, the generator automatically produces specific countermeasures.

[0081] As a concrete example, consider a case where a baby starts crying in the middle of the night. The device immediately sends various data to the server. The server uses a generative AI model to determine that "the baby is likely hungry" and generates a notification recommending breastfeeding. An example of a prompt that the user can actually input at this time would be in the format of "The baby is crying. Body temperature 37 degrees, room temperature 26 degrees, heart rate 120, and hunger signs detected from audio data. What is the recommended course of action?"

[0082] The generated countermeasures are sent to the device via a notification device and quickly communicated to the parents. Users can immediately take action based on the received information, and by feeding the results back into the system, the server improves the accuracy of the next analysis through its automatic learning function. This entire system allows parents to deal with their baby's nighttime crying more efficiently.

[0083] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0084] Step 1:

[0085] The device uses a wearable measuring device to acquire real-time biometric data such as the baby's body temperature, heart rate, and movement. Through this input data, the system aims to identify the baby's physiological state and detect abnormalities early. The collected biometric information is then transmitted to a server as output.

[0086] Step 2:

[0087] The device uses environmental sensors to collect environmental information such as room temperature, humidity, and sound pressure. By acquiring this environmental data, it becomes possible to determine whether the space where the baby is present is comfortable and safe. The input environmental information is visualized through a dashboard and transferred to the server as output.

[0088] Step 3:

[0089] The device uses a voice collection device to record the baby's cries and acquire audio information. This audio data serves as foundational data for analyzing the baby's emotions and needs. The input crying audio data is processed digitally to extract features, which are then sent to the server as output.

[0090] Step 4:

[0091] The server integrates biometric, environmental, and audio data, and uses advanced analytical equipment to assess the baby's psychological and physiological state. This analysis uses a generative AI model to analyze the correlations between each input data point and identify the primary causes of nighttime crying. The output is a list of the baby's estimated needs.

[0092] Step 5:

[0093] Based on the analysis results, the server uses a generator to produce specific countermeasures. These are recommended actions based on the analyzed data, and are notified to the user in a way that allows for immediate action. For example, the output might be an instruction such as, "Breastfeeding is recommended due to hunger."

[0094] Step 6:

[0095] The notification device sends the generated countermeasures to the user's terminal. The user receives this notification and can take appropriate action quickly. Based on the generated instructions, the user takes action and feeds the results back to the system.

[0096] Step 7:

[0097] The server receives feedback from users and uses an automated learning function to improve the accuracy of subsequent analyses. This feedback updates the generated AI model, resulting in improved efficiency and accuracy at each processing step.

[0098] (Application Example 1)

[0099] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0100] In today's childcare environment, parents spend a great deal of stress and time dealing with their babies' nighttime crying. Ensuring safety within the home is also a significant concern. Managing home security while simultaneously monitoring the baby's health and environmental comfort in real time has been difficult with conventional methods. This invention provides a multi-functional management system to solve these problems.

[0101] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0102] In this invention, the server includes a measurement means for acquiring biometric information, a detection means for acquiring environmental information, and a sound information collection means for acquiring audio information. This makes it possible to grasp the baby's health condition and environmental changes in real time, identify the cause of nighttime crying, and improve safety in the home by detecting abnormal sounds and movements.

[0103] "Biometric information" refers to numerical data that indicates the baby's physical condition, such as body temperature, heart rate, and movement.

[0104] "Measurement means" refers to devices and technologies used to accurately acquire biological information.

[0105] "Environmental information" refers to data that indicates external conditions that affect a baby's comfort, such as room temperature, humidity, and lighting levels.

[0106] "Detection means" refers to devices and technologies for collecting environmental information.

[0107] "Audio information" refers to data related to sound, including sounds such as a baby crying or unusual noises in the home.

[0108] "Audio information acquisition means" refers to devices or technologies that record audio information and provide it in a format that allows for analysis.

[0109] "Analysis means" refers to devices and technologies that integrate acquired biological information, environmental information, and audio information to comprehensively assess the baby's condition and the home environment.

[0110] "Generating means" refers to devices or technologies that create appropriate countermeasures or warnings based on analysis results.

[0111] "Notification means" refers to devices or technologies used to inform parents or users of generated countermeasures or warnings.

[0112] "Security analysis means" refers to devices and technologies that analyze audio information within a home, detect anomalies, and generate warnings.

[0113] A "warning notification means" refers to a device or technology used to send a warning to the user regarding a detected anomaly.

[0114] To implement this invention, the server needs to be equipped with measurement means for acquiring biometric information, detection means for acquiring environmental information, and sound information collection means for acquiring audio information. Each means can use wearable devices or sensors installed in the home. Specifically, a wearable device attached to the baby's body measures body temperature, heart rate, and movement in real time. Environmental information is acquired by temperature and humidity sensors and lighting sensors installed in the room. Audio information is collected by a terminal equipped with a microphone. All of this information is transmitted to the server.

[0115] The server integrates and analyzes this acquired information using an analysis tool. The analysis tool is data analysis software that utilizes a generative AI model and identifies patterns in the baby's cries through speech recognition. Furthermore, it combines body temperature, heart rate, movement, environmental information, and crying data to determine the baby's condition. For example, if a pattern indicating hunger is detected from the crying, it is compared with the ambient room temperature to generate appropriate countermeasures.

[0116] The countermeasures and warnings generated during this process are created by the generation mechanism and then notified to the parent or user on the device via the notification mechanism. Push notifications using smartphones are used for notifications, and the baby's condition and recommended actions (e.g., "Change the diaper" or "Feed the baby milk") are displayed intuitively.

[0117] Furthermore, if the home security analysis system detects an unusual sound, a warning is generated and immediately sent to the user via a warning notification system. Upon receiving this notification, the user can quickly check the security of their home.

[0118] As a concrete example, a user could enter the prompt "How can I simultaneously detect a baby crying at night and other unusual noises in the home?" and the system could then provide information about detecting crying or other unusual noises. The introduction of this system would make it possible to reduce the burden of childcare while maintaining a safe home environment.

[0119] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0120] Step 1:

[0121] The measurement method uses a wearable device to measure the baby's body temperature, heart rate, and movement. The input is biometric data, which is sent to a server as digital data. This data is acquired in real time and used by the server to assess the baby's health.

[0122] Step 2:

[0123] The detection system collects environmental data within the home through sensors. The input includes environmental data such as temperature, humidity, and lighting levels, and this information is also sent to a server. This is necessary to analyze how the environment affects the baby's comfort.

[0124] Step 3:

[0125] The sound information collection system uses the device's microphone to collect sounds such as a baby crying and other household noises. The input is an audio signal, which is converted into a digital format and sent to the server. The audio data is used to identify patterns in nighttime crying and unusual sounds within the home.

[0126] Step 4:

[0127] The server integrates and analyzes this acquired data using analytical tools. The input is all the data obtained in steps 1 to 3. Using a generative AI model, it detects crying patterns and environmental changes to make a comprehensive judgment about the baby's condition and the home environment. The output is the condition judgment result.

[0128] Step 5:

[0129] The generation unit generates countermeasures and warnings based on the judgment results from the analysis unit. The input is the judgment result from step 4. For example, specific actions such as "change the diaper" or "feed the baby milk" are instructed. The output is the generated countermeasures and warnings.

[0130] Step 6:

[0131] The notification system informs the user of the generated countermeasures and warnings via the device. The input is the output from step 5. Information is delivered to the user intuitively via push notifications using a smartphone.

[0132] Step 7:

[0133] The security analysis system analyzes unusual sounds within the home and generates warnings as needed. The input is the audio data obtained in step 3. To ensure reliable security measures are taken, the server reacts immediately upon detecting an unusual sound. The output is the generated security warning.

[0134] Step 8:

[0135] The warning notification system notifies the user of security alerts. The input is the output of step 7. In cases of high urgency, a higher warning level is used to quickly draw the user's attention.

[0136] These steps make it possible to efficiently manage both the baby's health and home security during childcare.

[0137] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0138] This invention relates to a support system for reducing nighttime crying in babies and parental stress during childcare. The system includes a measuring device, a sensor device, a voice collection device, an analysis device, a generation device, a notification device, and an emotion engine, which enable the provision of coping strategies that take into account the user's emotional state.

[0139] First, the device acquires the baby's biometric information using a wearable measuring device. This is used to monitor body temperature, heart rate, movement, etc., in real time. In addition, the device collects temperature and humidity from installed sensor devices to obtain environmental information.

[0140] Next, the device records the baby's cries using a voice collection device and saves it as audio data. Based on this audio data, a voice recognition algorithm on the server analyzes the crying patterns and infers the baby's state, such as hunger, discomfort, or sleepiness.

[0141] In this invention, a newly introduced emotion engine on the server plays a major role. This emotion engine uses acquired voice data and parent biometric data to identify the parent's emotional state. For example, it can assess whether the parent is stressed or relaxed based on voice tone and biometric information (such as changes in heart rate).

[0142] The server integrates this data and uses an analysis device to comprehensively assess the state of the baby and parents. Based on this assessment, the generator takes the output of the emotion engine into account when generating coping strategies based on the identified causes. This allows for the recommendation of alternative approaches, for example, when parents are under stress.

[0143] The generated response measures are sent to the device via a notification system, and the parents are shown recommendations. These notifications include specific action plans, such as "Your baby may be hungry, please breastfeed them" or "Your stress levels are high, so we recommend taking a few minutes to breathe deeply."

[0144] Users can take appropriate action based on this notification and provide feedback on its effectiveness to the system. This feedback is sent to the server, and the system continuously improves through its automated learning function, enabling it to provide more accurate and personalized support.

[0145] This system allows parents to monitor their own emotional state while caring for their baby, reducing parenting stress and enabling them to parent more effectively.

[0146] The following describes the processing flow.

[0147] Step 1:

[0148] The device uses a wearable measuring device to acquire biometric data such as the baby's body temperature, heart rate, and movement in real time.

[0149] Step 2:

[0150] The terminal uses sensor devices installed in the room to collect environmental data such as temperature and humidity.

[0151] Step 3:

[0152] The device uses a voice collection device to record the baby's cries and generates audio data.

[0153] Step 4:

[0154] The terminal transmits all of the above acquired data to the server via the communication module.

[0155] Step 5:

[0156] The server analyzes the audio data and uses a speech recognition algorithm to identify crying patterns and infer the state of the crying.

[0157] Step 6:

[0158] The server integrates biometric and environmental data, and, along with the results of the crying analysis, assesses the baby's overall condition.

[0159] Step 7:

[0160] The server uses an emotion engine to analyze the user's voice tone and biometric data to identify the user's emotional state.

[0161] Step 8:

[0162] The server comprehensively evaluates the baby's and the user's condition and uses a generator to produce the optimal course of action. The user's emotional state is taken into consideration during this process.

[0163] Step 9:

[0164] The server sends the generated corrective actions to the terminal.

[0165] Step 10:

[0166] The device will notify the user of recommendations. These notifications will include specific advice on baby care and mental support for the user.

[0167] Step 11:

[0168] The user takes action based on the notification content and returns the results and feedback to the system via their device.

[0169] Step 12:

[0170] The server analyzes user feedback and uses automated learning algorithms to improve the system's recommendation accuracy and sentiment recognition accuracy.

[0171] (Example 2)

[0172] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0173] In childcare, nighttime crying of babies and parental stress are major challenges, and many parents experience anxiety and fatigue. This invention provides appropriate countermeasures based on the baby's biological information and crying, as well as the parent's biological information, thereby reducing the stress of childcare and enabling more effective childcare.

[0174] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0175] In this invention, the server includes measuring means for acquiring biological information, sensor means for acquiring environmental information, and voice acquisition means for acquiring voice information. This makes it possible to accurately evaluate the condition of the baby and the parent and to provide the parent with specific and useful countermeasures.

[0176] "Biometric information" refers to information obtained from the body of a baby or parent, such as body temperature, heart rate, and body movements.

[0177] "Environmental information" refers to information related to the environment of a given location, such as temperature and humidity.

[0178] "Audio information" refers to audio data that includes sounds such as a baby crying or other ambient sounds.

[0179] "Analysis means" refers to a processing mechanism that integrates acquired biological information, environmental information, and audio information to evaluate the state of the subject.

[0180] A "generating mechanism" is a mechanism that creates coping measures using an emotion evaluation mechanism based on the results of an analysis.

[0181] "Notification means" refers to a means of informing parents of the countermeasures generated by the generation means.

[0182] The "emotional evaluation mechanism" is a function that evaluates the emotional state of parents through auditory and biometric information.

[0183] The "automatic learning function" is a continuous learning function that uses user feedback to improve the accuracy of the system.

[0184] This invention is a system for parents to effectively manage their baby's nighttime crying and their own stress. This system primarily consists of three elements: a terminal, a server, and a user.

[0185] The device acquires the baby's biometric information through wearable measurement devices. Specifically, it uses devices such as smart bands and baby monitors to collect the baby's body temperature, heart rate, and movement in real time. It also uses sensor devices to collect environmental information such as room temperature and humidity.

[0186] The device uses a microphone-equipped voice collection device to record the baby's cries and saves them as audio data. This data is then sent to a server equipped with a data analysis algorithm.

[0187] The server uses speech recognition algorithms and emotion evaluation mechanisms to assess the state of the baby and parent. The audio data is analyzed using machine learning libraries such as TENSORFLOW® and PyTorch to infer patterns in the baby's cries and the parent's emotional state.

[0188] The server's generation device generates appropriate countermeasures based on the evaluation results. It utilizes a generation AI model and is implemented in programming languages ​​such as Python. The generated countermeasures are sent to the device via a notification device, providing parents with specific action guidelines. For example, notifications such as "Your baby may be hungry, so we recommend breastfeeding" or "Your stress levels are high, so we recommend taking a few minutes to breathe deeply" may be sent.

[0189] Users can improve the efficiency of childcare and their own mental state by taking action based on notifications. User feedback is sent to the server and used by an automated learning function to improve the accuracy of the system.

[0190] As a concrete example of a prompt, one could ask the generating AI model, "What are some ways to reduce parental stress when a baby is crying?" In response to this inquiry, the system would suggest the most appropriate course of action.

[0191] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0192] Step 1:

[0193] The device acquires biometric information such as the baby's body temperature, heart rate, and movement using a wearable measuring device. The input is real-time sensor data obtained from the wearable device, which is then converted to a digital format for output. Specifically, biometric information is continuously transmitted when the smart band is worn.

[0194] Step 2:

[0195] The terminal uses sensor devices installed in the room to acquire environmental information such as temperature and humidity. The input is real-time data from the environmental sensors, which is then converted to a digital format and output. In operation, the sensor devices periodically measure the indoor environmental data and store it in a database.

[0196] Step 3:

[0197] The device records the baby's cries using a voice collection device. The input is an analog audio signal from a voice sensor, which is converted into digital audio data and output. Specifically, the microphone detects the baby's cries and saves them as an audio file.

[0198] Step 4:

[0199] The server analyzes the collected audio data using a speech recognition algorithm. The input is digital audio data, and it analyzes the audio patterns to generate output that estimates the cause of crying (hunger, discomfort, sleepiness, etc.). In terms of operation, it applies a model using machine learning libraries such as TensorFlow or PyTorch and analyzes the results.

[0200] Step 5:

[0201] The server identifies the parent's emotional state using voice data and the parent's biometric data. The input consists of changes in voice tone and biometric information, generating an output that evaluates emotional states such as stress and relaxation. Specifically, data analysis is performed through an emotion evaluation mechanism.

[0202] Step 6:

[0203] Based on the aforementioned analysis results, the server generates specific countermeasures through a generation device. The input is the assessment results of the baby's and parent's condition, and based on this, it outputs countermeasures including recommended actions. In operation, the optimal action plan is dynamically created using a generation AI model.

[0204] Step 7:

[0205] The notification device sends the generated corrective actions to the device and displays them to the parent. The input is corrective action guideline data, which is output as a user-friendly notification. Specifically, information such as "Your baby may be hungry, so we recommend breastfeeding" is visually displayed on the device screen.

[0206] Step 8:

[0207] Users take action based on the notified guidelines and provide feedback to the system regarding the results. The input consists of the user's actual actions and their outcomes, which the system receives and adds to its learning database as output. Through this feedback mechanism, the system continuously improves.

[0208] (Application Example 2)

[0209] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0210] In many workplaces, properly managing the physical and mental stress of workers is a challenge. Accumulated stress among workers can lead to decreased work efficiency and health problems, making real-time stress assessment and effective countermeasures necessary.

[0211] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0212] In this invention, the server includes measuring means for acquiring biological information, detection means for acquiring environmental information, and collection means for acquiring sound wave information. This makes it possible to monitor the physical and mental state of workers in real time and recommend appropriate measures.

[0213] "Biometric information" refers to data that indicates the physical condition of a worker, such as their heart rate and body temperature.

[0214] "Environmental information" refers to data that indicates external conditions of the work environment, such as temperature and humidity.

[0215] "Sound wave information" refers to data related to voice and other sounds, which is acquired to understand the conditions of workers and the environment.

[0216] "Measurement means" refers to devices and sensors used to acquire the biometric information of workers.

[0217] "Detection means" refers to devices or sensors used to acquire information about the work environment.

[0218] "Collection means" refers to devices or systems for acquiring sound wave information.

[0219] "Analysis means" refers to algorithms and systems that integrate acquired data to evaluate the worker's condition.

[0220] "Generation means" refers to a device or program that creates recommended countermeasures based on the analysis results.

[0221] A "notification means" is a device or system used to communicate the generated countermeasures to workers.

[0222] The system for realizing this application example operates through the collaboration of a server, a terminal, and a user. The server receives biometric information, environmental information, and sound wave information transmitted from the measurement means, detection means, and collection means. The measurement means acquires the worker's biometric information in real time using hardware such as heart rate sensors and body temperature sensors. The detection means acquires information about the work environment using temperature sensors and humidity sensors. The collection means acquires sound wave information using speech recognition technology.

[0223] The server integrates the acquired data and evaluates the worker's condition through analysis. This analysis uses speech recognition algorithms and machine learning models for stress detection. For example, the server might determine that a worker's stress level is increasing if their heart rate is higher than normal.

[0224] Based on the analysis results, the generation system proposes appropriate measures to the worker. These measures may include voice notifications or visual instructions via a display. For example, it might recommend that a worker experiencing high stress levels "take a 5-minute break."

[0225] Users manage their stress by accepting and implementing these suggestions via notification mechanisms. User feedback is sent to the server and used as feedback to an automated learning model to improve the accuracy of recommendations.

[0226] Examples of prompt statements to input into the generative AI model are as follows:

[0227] "Developer prompt: Please show us how to build an AI system that suggests measures to reduce worker stress in a factory environment. Please also provide example code that generates appropriate notifications based on biometric data (heart rate, body temperature) and environmental data."

[0228] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0229] Step 1:

[0230] The terminal acquires biometric information such as heart rate and body temperature through the wearable device worn by the worker. This biometric information becomes the input data. The terminal collects this data using a sensor API and prepares to send it to the server.

[0231] Step 2:

[0232] The terminal acquires ambient environmental information using installed temperature and humidity sensors. This environmental information becomes the input data. The acquired temperature and humidity data is then organized for transmission to the server.

[0233] Step 3:

[0234] The terminal records sound wave information, particularly the worker's voice or ambient sounds, through a collection device. This sound wave data becomes the input data. The recorded audio data is converted to an audio format and prepared for transfer to the server.

[0235] Step 4:

[0236] The server integrates the received biometric, environmental, and acoustic information. Here, data in different formats is stored in a common database and prepared for the next analysis step. Data integrity is checked, and missing values ​​are imputed.

[0237] Step 5:

[0238] The server processes the integrated data using analytical tools. This analysis includes a stress detection algorithm. Based on the input data, it evaluates the worker's stress level and generates an output indicating high or low stress levels.

[0239] Step 6:

[0240] The server generates suggestions using a generation mechanism based on the analyzed results. In this step, recommendations for rest and exercise tailored to the stress level are specified and output as notification messages.

[0241] Step 7:

[0242] The user receives suggestions via a notification method. The suggestions appear on the device as pop-ups or voice messages. The user selects an action based on the recommendation and enters feedback on the device.

[0243] Step 8:

[0244] The server receives feedback from users. This feedback data is used for subsequent data analysis and improvement of the accuracy of suggestions. The automated learning model is updated and the changes are reflected in future analyses.

[0245] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0246] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0247] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0248] [Second Embodiment]

[0249] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0250] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0251] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0252] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0253] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0254] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0255] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0256] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0257] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0258] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0259] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0260] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0261] This invention is a support system for solving the problem of nighttime crying in infants during childcare, and includes a measuring device, a sensor device, a sound collection device, an analysis device, a generation device, and a notification device. This system has the function of collecting and integrating the infant's biological data, surrounding environment data, and crying sound data, and analyzing this data to identify the cause of nighttime crying and notify parents of appropriate countermeasures.

[0262] First, the device uses a wearable measuring device to acquire information about the baby's body temperature, heart rate, and movement. This information is usually acquired in real time and used to monitor the baby's health and stability. The device also acquires environmental information such as temperature and humidity using sensors installed in the room. This allows the device to constantly know whether the room is a comfortable environment for the baby.

[0263] The terminal's voice collection device records the baby's cries and generates audio data. This audio data is an important source of information for analyzing crying patterns. The obtained crying data is analyzed by a voice recognition algorithm, and the psychological and physiological state corresponding to the crying pattern is inferred.

[0264] Next, the server integrates this data and uses an analysis device to make a comprehensive assessment of the baby's condition. For example, even if the crying pattern indicates "hunger," it compares this data with other information such as body temperature and ambient temperature to determine the most appropriate course of action.

[0265] Based on the analysis results, the generator produces specific countermeasures. These countermeasures are presented in an intuitive and easy-to-understand way for parents, such as, "It is highly likely that the child is hungry, so we recommend breastfeeding."

[0266] The generated countermeasures are notified to the parent from the device via a notification device. The user can then take appropriate action immediately based on this notification. Furthermore, the parent's response is input into the system as feedback, which the server analyzes and uses its automatic learning function to improve the accuracy of future recommendations. This enables the continuous provision of personalized support.

[0267] The introduction of this system will allow parents to deal with their baby's nighttime crying problem with less stress and less time, and as a result, a significant improvement in the childcare environment is expected.

[0268] The following describes the processing flow.

[0269] Step 1:

[0270] The device uses a wearable measuring device to acquire biometric data such as the baby's body temperature, heart rate, and movement in real time.

[0271] Step 2:

[0272] The terminal uses sensor devices installed in the room to collect environmental data such as temperature, humidity, and lighting intensity.

[0273] Step 3:

[0274] The device uses a voice collection device to record the baby's cries and generate audio data.

[0275] Step 4:

[0276] The terminal sends all the data acquired in steps 1, 2, and 3 to the server via the communication module.

[0277] Step 5:

[0278] The server analyzes the received audio data, uses a speech recognition algorithm to identify crying patterns, and infers the state (for example, finding signs of hunger or discomfort).

[0279] Step 6:

[0280] The server integrates biometric and environmental data, and performs data analysis to assess the baby's overall condition, along with the results of the crying analysis.

[0281] Step 7:

[0282] Based on the analysis results, the server generates the optimal countermeasures for the identified causes via a generator.

[0283] Step 8:

[0284] The server transmits the countermeasures generated to the terminal.

[0285] Step 9:

[0286] The terminal notifies the parent and presents the recommended countermeasures (e.g., "There may be hunger. Breastfeeding is recommended") in real time.

[0287] Step 10:

[0288] The user responds to the baby based on the notification and inputs the results and feedback into the system through the terminal.

[0289] Step 11:

[0290] The server analyzes the feedback from the user, updates the data analysis model using an automatic learning algorithm, and aims to improve the accuracy of future countermeasure proposals.

[0291] (Example 1)

[0292] Next, Example 1 will be described. In the following description, the data processing device 12 is referred to as the "server", and the smart glasses 214 are referred to as the "terminal".

[0293] Night crying of a baby in childcare is a major stress and burden for parents. With conventional methods, it is difficult to identify and address the cause of the baby's crying, and it is often hard to find an appropriate method. Therefore, there is a demand for a system that can accurately grasp the cause of the baby's crying and quickly take appropriate countermeasures.

[0294] The specific processing by the specific processing unit 290 of the data processing device 12 in Example 1 is realized by the following means. <000093​​In this invention, the server includes measuring means for acquiring biological information, detection means for acquiring environmental information, and sound collection means for acquiring sound information. This enables comprehensive data analysis of the baby's physiological state and surrounding environment, allowing for the identification of the cause of the baby's crying and the rapid presentation of appropriate countermeasures to the parents.

[0296] "Biometric information" refers to data related to a baby's physical condition and reactions, such as body temperature, heart rate, and movement.

[0297] "Environmental information" refers to data related to the surrounding environmental conditions, such as temperature, humidity, and sound pressure in the space where the baby is located.

[0298] "Auditory information" refers to information expressed through sound, such as a baby crying.

[0299] "Measuring means" refers to devices and methods for acquiring biological information, which allows for monitoring the baby's health.

[0300] "Detection means" refers to devices and methods for sensing and acquiring environmental information, which allows us to obtain information to determine the comfort level of the environment in which a baby is present.

[0301] "Voice collection means" refers to devices and methods for acquiring voice information, which can be used to obtain data for analyzing patterns in a baby's cries.

[0302] "Analysis means" refers to devices and methods for integrating and analyzing acquired biological information, environmental information, and audio information to evaluate the baby's condition.

[0303] "Generating means" refers to devices or methods that create recommended countermeasures based on analyzed data.

[0304] The "notification means" refers to a device or method for notifying a user such as a parent of the generated countermeasures, enabling prompt response.

[0305] The present invention is a support system for solving the problem of a baby crying at night. This system is mainly composed of a server, a terminal, and a user. It acquires biological information, environmental information, and voice information, integrates and analyzes each data, evaluates the baby's state, generates appropriate countermeasures, and notifies them.

[0306] The terminal acquires biological information such as the baby's body temperature, heart rate, and movement in real time through a wearable measurement device. Also, it collects environmental information such as the room temperature and humidity using an environmental sensor. This enables constant monitoring of whether the baby's living space is comfortable. Furthermore, the voice collection device records the baby's crying sound and acquires voice information. These various data are transmitted to the server via a network.

[0307] The server integrates the received data and comprehensively evaluates the baby's psychological and physiological state using an advanced analysis device. A generation AI model is used for this analysis, for example, analyzing the characteristics of the crying sound to infer causes such as "hungry" or "unhappy". Based on the results of the analysis, a generation device automatically generates specific countermeasures.

[0308] As a specific example, consider the case where the baby starts crying at night. The terminal immediately transmits various data to the server. The server uses the generation AI model to determine that "the baby is likely hungry" and generates a notification recommending breastfeeding. At this time, an example of the prompt text that the user can actually input is in the form of "The baby is crying. Body temperature is 37 degrees, room temperature is 26 degrees, heart rate is 120, and a hunger sign has been detected from the voice data. What is the recommended response?".

[0309] The generated countermeasures are sent to the device via a notification device and quickly communicated to the parents. Users can immediately take action based on the received information, and by feeding the results back into the system, the server improves the accuracy of the next analysis through its automatic learning function. This entire system allows parents to deal with their baby's nighttime crying more efficiently.

[0310] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0311] Step 1:

[0312] The device uses a wearable measuring device to acquire real-time biometric data such as the baby's body temperature, heart rate, and movement. Through this input data, the system aims to identify the baby's physiological state and detect abnormalities early. The collected biometric information is then transmitted to a server as output.

[0313] Step 2:

[0314] The device uses environmental sensors to collect environmental information such as room temperature, humidity, and sound pressure. By acquiring this environmental data, it becomes possible to determine whether the space where the baby is present is comfortable and safe. The input environmental information is visualized through a dashboard and transferred to the server as output.

[0315] Step 3:

[0316] The device uses a voice collection device to record the baby's cries and acquire audio information. This audio data serves as foundational data for analyzing the baby's emotions and needs. The input crying audio data is processed digitally to extract features, which are then sent to the server as output.

[0317] Step 4:

[0318] The server integrates biometric, environmental, and audio data, and uses advanced analytical equipment to assess the baby's psychological and physiological state. This analysis uses a generative AI model to analyze the correlations between each input data point and identify the primary causes of nighttime crying. The output is a list of the baby's estimated needs.

[0319] Step 5:

[0320] Based on the analysis results, the server uses a generator to produce specific countermeasures. These are recommended actions based on the analyzed data, and are notified to the user in a way that allows for immediate action. For example, the output might be an instruction such as, "Breastfeeding is recommended due to hunger."

[0321] Step 6:

[0322] The notification device sends the generated countermeasures to the user's terminal. The user receives this notification and can take appropriate action quickly. Based on the generated instructions, the user takes action and feeds the results back to the system.

[0323] Step 7:

[0324] The server receives feedback from users and uses an automated learning function to improve the accuracy of subsequent analyses. This feedback updates the generated AI model, resulting in improved efficiency and accuracy at each processing step.

[0325] (Application Example 1)

[0326] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0327] In today's childcare environment, parents spend a great deal of stress and time dealing with their babies' nighttime crying. Ensuring safety within the home is also a significant concern. Managing home security while simultaneously monitoring the baby's health and environmental comfort in real time has been difficult with conventional methods. This invention provides a multi-functional management system to solve these problems.

[0328] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0329] In this invention, the server includes a measurement means for acquiring biometric information, a detection means for acquiring environmental information, and a sound information collection means for acquiring audio information. This makes it possible to grasp the baby's health condition and environmental changes in real time, identify the cause of nighttime crying, and improve safety in the home by detecting abnormal sounds and movements.

[0330] "Biometric information" refers to numerical data that indicates the baby's physical condition, such as body temperature, heart rate, and movement.

[0331] "Measurement means" refers to devices and technologies used to accurately acquire biological information.

[0332] "Environmental information" refers to data that indicates external conditions that affect a baby's comfort, such as room temperature, humidity, and lighting levels.

[0333] "Detection means" refers to devices and technologies for collecting environmental information.

[0334] "Audio information" refers to data related to sound, including sounds such as a baby crying or unusual noises in the home.

[0335] "Audio information acquisition means" refers to devices or technologies that record audio information and provide it in a format that allows for analysis.

[0336] "Analysis means" refers to devices and technologies that integrate acquired biological information, environmental information, and audio information to comprehensively assess the baby's condition and the home environment.

[0337] "Generating means" refers to devices or technologies that create appropriate countermeasures or warnings based on analysis results.

[0338] "Notification means" refers to devices or technologies used to inform parents or users of generated countermeasures or warnings.

[0339] "Security analysis means" refers to devices and technologies that analyze audio information within a home, detect anomalies, and generate warnings.

[0340] A "warning notification means" refers to a device or technology used to send a warning to the user regarding a detected anomaly.

[0341] To implement this invention, the server needs to be equipped with measurement means for acquiring biometric information, detection means for acquiring environmental information, and sound information collection means for acquiring audio information. Each means can use wearable devices or sensors installed in the home. Specifically, a wearable device attached to the baby's body measures body temperature, heart rate, and movement in real time. Environmental information is acquired by temperature and humidity sensors and lighting sensors installed in the room. Audio information is collected by a terminal equipped with a microphone. All of this information is transmitted to the server.

[0342] The server integrates and analyzes this acquired information using an analysis tool. The analysis tool is data analysis software that utilizes a generative AI model and identifies patterns in the baby's cries through speech recognition. Furthermore, it combines body temperature, heart rate, movement, environmental information, and crying data to determine the baby's condition. For example, if a pattern indicating hunger is detected from the crying, it is compared with the ambient room temperature to generate appropriate countermeasures.

[0343] The countermeasures and warnings generated during this process are created by the generation mechanism and then notified to the parent or user on the device via the notification mechanism. Push notifications using smartphones are used for notifications, and the baby's condition and recommended actions (e.g., "Change the diaper" or "Feed the baby milk") are displayed intuitively.

[0344] Furthermore, if the home security analysis system detects an unusual sound, a warning is generated and immediately sent to the user via a warning notification system. Upon receiving this notification, the user can quickly check the security of their home.

[0345] As a concrete example, a user could enter the prompt "How can I simultaneously detect a baby crying at night and other unusual noises in the home?" and the system could then provide information about detecting crying or other unusual noises. The introduction of this system would make it possible to reduce the burden of childcare while maintaining a safe home environment.

[0346] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0347] Step 1:

[0348] The measurement method uses a wearable device to measure the baby's body temperature, heart rate, and movement. The input is biometric data, which is sent to a server as digital data. This data is acquired in real time and used by the server to assess the baby's health.

[0349] Step 2:

[0350] The detection system collects environmental data within the home through sensors. The input includes environmental data such as temperature, humidity, and lighting levels, and this information is also sent to a server. This is necessary to analyze how the environment affects the baby's comfort.

[0351] Step 3:

[0352] The sound information collection system uses the device's microphone to collect sounds such as a baby crying and other household noises. The input is an audio signal, which is converted into a digital format and sent to the server. The audio data is used to identify patterns in nighttime crying and unusual sounds within the home.

[0353] Step 4:

[0354] The server integrates and analyzes this acquired data using analytical tools. The input is all the data obtained in steps 1 to 3. Using a generative AI model, it detects crying patterns and environmental changes to make a comprehensive judgment about the baby's condition and the home environment. The output is the condition judgment result.

[0355] Step 5:

[0356] The generation unit generates countermeasures and warnings based on the judgment results from the analysis unit. The input is the judgment result from step 4. For example, specific actions such as "change the diaper" or "feed the baby milk" are instructed. The output is the generated countermeasures and warnings.

[0357] Step 6:

[0358] The notification system informs the user of the generated countermeasures and warnings via the device. The input is the output from step 5. Information is delivered to the user intuitively via push notifications using a smartphone.

[0359] Step 7:

[0360] The security analysis system analyzes unusual sounds within the home and generates warnings as needed. The input is the audio data obtained in step 3. To ensure reliable security measures are taken, the server reacts immediately upon detecting an unusual sound. The output is the generated security warning.

[0361] Step 8:

[0362] The warning notification system notifies the user of security alerts. The input is the output of step 7. In cases of high urgency, a higher warning level is used to quickly draw the user's attention.

[0363] These steps make it possible to efficiently manage both the baby's health and home security during childcare.

[0364] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0365] This invention relates to a support system for reducing nighttime crying in babies and parental stress during childcare. The system includes a measuring device, a sensor device, a voice collection device, an analysis device, a generation device, a notification device, and an emotion engine, which enable the provision of coping strategies that take into account the user's emotional state.

[0366] First, the device acquires the baby's biometric information using a wearable measuring device. This is used to monitor body temperature, heart rate, movement, etc., in real time. In addition, the device collects temperature and humidity from installed sensor devices to obtain environmental information.

[0367] Next, the device records the baby's cries using a voice collection device and saves it as audio data. Based on this audio data, a voice recognition algorithm on the server analyzes the crying patterns and infers the baby's state, such as hunger, discomfort, or sleepiness.

[0368] In this invention, a newly introduced emotion engine on the server plays a major role. This emotion engine uses acquired voice data and parent biometric data to identify the parent's emotional state. For example, it can assess whether the parent is stressed or relaxed based on voice tone and biometric information (such as changes in heart rate).

[0369] The server integrates this data and uses an analysis device to comprehensively assess the state of the baby and parents. Based on this assessment, the generator takes the output of the emotion engine into account when generating coping strategies based on the identified causes. This allows for the recommendation of alternative approaches, for example, when parents are under stress.

[0370] The generated response measures are sent to the device via a notification system, and the parents are shown recommendations. These notifications include specific action plans, such as "Your baby may be hungry, please breastfeed them" or "Your stress levels are high, so we recommend taking a few minutes to breathe deeply."

[0371] Users can take appropriate action based on this notification and provide feedback on its effectiveness to the system. This feedback is sent to the server, and the system continuously improves through its automated learning function, enabling it to provide more accurate and personalized support.

[0372] This system allows parents to monitor their own emotional state while caring for their baby, reducing parenting stress and enabling them to parent more effectively.

[0373] The following describes the processing flow.

[0374] Step 1:

[0375] The device uses a wearable measuring device to acquire biometric data such as the baby's body temperature, heart rate, and movement in real time.

[0376] Step 2:

[0377] The terminal uses sensor devices installed in the room to collect environmental data such as temperature and humidity.

[0378] Step 3:

[0379] The device uses a voice collection device to record the baby's cries and generates audio data.

[0380] Step 4:

[0381] The terminal transmits all of the above acquired data to the server via the communication module.

[0382] Step 5:

[0383] The server analyzes the audio data and uses a speech recognition algorithm to identify crying patterns and infer the state of the crying.

[0384] Step 6:

[0385] The server integrates biometric and environmental data, and, along with the results of the crying analysis, assesses the baby's overall condition.

[0386] Step 7:

[0387] The server uses an emotion engine to analyze the user's voice tone and biometric data to identify the user's emotional state.

[0388] Step 8:

[0389] The server comprehensively evaluates the baby's and the user's condition and uses a generator to produce the optimal course of action. The user's emotional state is taken into consideration during this process.

[0390] Step 9:

[0391] The server sends the generated corrective actions to the terminal.

[0392] Step 10:

[0393] The device will notify the user of recommendations. These notifications will include specific advice on baby care and mental support for the user.

[0394] Step 11:

[0395] The user takes action based on the notification content and returns the results and feedback to the system via their device.

[0396] Step 12:

[0397] The server analyzes user feedback and uses automated learning algorithms to improve the system's recommendation accuracy and sentiment recognition accuracy.

[0398] (Example 2)

[0399] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0400] In childcare, nighttime crying of babies and parental stress are major challenges, and many parents experience anxiety and fatigue. This invention provides appropriate countermeasures based on the baby's biological information and crying, as well as the parent's biological information, thereby reducing the stress of childcare and enabling more effective childcare.

[0401] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0402] In this invention, the server includes measuring means for acquiring biological information, sensor means for acquiring environmental information, and voice acquisition means for acquiring voice information. This makes it possible to accurately evaluate the condition of the baby and the parent and to provide the parent with specific and useful countermeasures.

[0403] "Biometric information" refers to information obtained from the body of a baby or parent, such as body temperature, heart rate, and body movements.

[0404] "Environmental information" refers to information related to the environment of a given location, such as temperature and humidity.

[0405] "Audio information" refers to audio data that includes sounds such as a baby crying or other ambient sounds.

[0406] "Analysis means" refers to a processing mechanism that integrates acquired biological information, environmental information, and audio information to evaluate the state of the subject.

[0407] A "generating mechanism" is a mechanism that creates coping measures using an emotion evaluation mechanism based on the results of an analysis.

[0408] "Notification means" refers to a means of informing parents of the countermeasures generated by the generation means.

[0409] The "emotional evaluation mechanism" is a function that evaluates the emotional state of parents through auditory and biometric information.

[0410] The "automatic learning function" is a continuous learning function that uses user feedback to improve the accuracy of the system.

[0411] This invention is a system for parents to effectively manage their baby's nighttime crying and their own stress. This system primarily consists of three elements: a terminal, a server, and a user.

[0412] The device acquires the baby's biometric information through wearable measurement devices. Specifically, it uses devices such as smart bands and baby monitors to collect the baby's body temperature, heart rate, and movement in real time. It also uses sensor devices to collect environmental information such as room temperature and humidity.

[0413] The device uses a microphone-equipped voice collection device to record the baby's cries and saves them as audio data. This data is then sent to a server equipped with a data analysis algorithm.

[0414] The server uses speech recognition algorithms and emotion evaluation mechanisms to assess the state of the baby and parent. The audio data is analyzed using machine learning libraries such as TensorFlow and PyTorch to infer patterns in the baby's cries and the parent's emotional state.

[0415] The server's generation device generates appropriate countermeasures based on the evaluation results. It utilizes a generation AI model and is implemented in programming languages ​​such as Python. The generated countermeasures are sent to the device via a notification device, providing parents with specific action guidelines. For example, notifications such as "Your baby may be hungry, so we recommend breastfeeding" or "Your stress levels are high, so we recommend taking a few minutes to breathe deeply" may be sent.

[0416] Users can improve the efficiency of childcare and their own mental state by taking action based on notifications. User feedback is sent to the server and used by an automated learning function to improve the accuracy of the system.

[0417] As a concrete example of a prompt, one could ask the generating AI model, "What are some ways to reduce parental stress when a baby is crying?" In response to this inquiry, the system would suggest the most appropriate course of action.

[0418] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0419] Step 1:

[0420] The device acquires biometric information such as the baby's body temperature, heart rate, and movement using a wearable measuring device. The input is real-time sensor data obtained from the wearable device, which is then converted to a digital format for output. Specifically, biometric information is continuously transmitted when the smart band is worn.

[0421] Step 2:

[0422] The terminal uses sensor devices installed in the room to acquire environmental information such as temperature and humidity. The input is real-time data from the environmental sensors, which is then converted to a digital format and output. In operation, the sensor devices periodically measure the indoor environmental data and store it in a database.

[0423] Step 3:

[0424] The device records the baby's cries using a voice collection device. The input is an analog audio signal from a voice sensor, which is converted into digital audio data and output. Specifically, the microphone detects the baby's cries and saves them as an audio file.

[0425] Step 4:

[0426] The server analyzes the collected audio data using a speech recognition algorithm. The input is digital audio data, and it analyzes the audio patterns to generate output that estimates the cause of crying (hunger, discomfort, sleepiness, etc.). In terms of operation, it applies a model using machine learning libraries such as TensorFlow or PyTorch and analyzes the results.

[0427] Step 5:

[0428] The server identifies the parent's emotional state using voice data and the parent's biometric data. The input consists of changes in voice tone and biometric information, generating an output that evaluates emotional states such as stress and relaxation. Specifically, data analysis is performed through an emotion evaluation mechanism.

[0429] Step 6:

[0430] Based on the aforementioned analysis results, the server generates specific countermeasures through a generation device. The input is the assessment results of the baby's and parent's condition, and based on this, it outputs countermeasures including recommended actions. In operation, the optimal action plan is dynamically created using a generation AI model.

[0431] Step 7:

[0432] The notification device sends the generated corrective actions to the device and displays them to the parent. The input is corrective action guideline data, which is output as a user-friendly notification. Specifically, information such as "Your baby may be hungry, so we recommend breastfeeding" is visually displayed on the device screen.

[0433] Step 8:

[0434] Users take action based on the notified guidelines and provide feedback to the system regarding the results. The input consists of the user's actual actions and their outcomes, which the system receives and adds to its learning database as output. Through this feedback mechanism, the system continuously improves.

[0435] (Application Example 2)

[0436] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0437] In many workplaces, properly managing the physical and mental stress of workers is a challenge. Accumulated stress among workers can lead to decreased work efficiency and health problems, making real-time stress assessment and effective countermeasures necessary.

[0438] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0439] In this invention, the server includes measuring means for acquiring biological information, detection means for acquiring environmental information, and collection means for acquiring sound wave information. This makes it possible to monitor the physical and mental state of workers in real time and recommend appropriate measures.

[0440] "Biometric information" refers to data that indicates the physical condition of a worker, such as their heart rate and body temperature.

[0441] "Environmental information" refers to data that indicates external conditions of the work environment, such as temperature and humidity.

[0442] "Sound wave information" refers to data related to voice and other sounds, which is acquired to understand the conditions of workers and the environment.

[0443] "Measurement means" refers to devices and sensors used to acquire the biometric information of workers.

[0444] "Detection means" refers to devices or sensors used to acquire information about the work environment.

[0445] "Collection means" refers to devices or systems for acquiring sound wave information.

[0446] "Analysis means" refers to algorithms and systems that integrate acquired data to evaluate the worker's condition.

[0447] "Generation means" refers to a device or program that creates recommended countermeasures based on the analysis results.

[0448] A "notification means" is a device or system used to communicate the generated countermeasures to workers.

[0449] The system for realizing this application example operates through the collaboration of a server, a terminal, and a user. The server receives biometric information, environmental information, and sound wave information transmitted from the measurement means, detection means, and collection means. The measurement means acquires the worker's biometric information in real time using hardware such as heart rate sensors and body temperature sensors. The detection means acquires information about the work environment using temperature sensors and humidity sensors. The collection means acquires sound wave information using speech recognition technology.

[0450] The server integrates the acquired data and evaluates the worker's condition through analysis. This analysis uses speech recognition algorithms and machine learning models for stress detection. For example, the server might determine that a worker's stress level is increasing if their heart rate is higher than normal.

[0451] Based on the analysis results, the generation system proposes appropriate measures to the worker. These measures may include voice notifications or visual instructions via a display. For example, it might recommend that a worker experiencing high stress levels "take a 5-minute break."

[0452] Users manage their stress by accepting and implementing these suggestions via notification mechanisms. User feedback is sent to the server and used as feedback to an automated learning model to improve the accuracy of recommendations.

[0453] Examples of prompt statements to input into the generative AI model are as follows:

[0454] "Developer prompt: Please show us how to build an AI system that suggests measures to reduce worker stress in a factory environment. Please also provide example code that generates appropriate notifications based on biometric data (heart rate, body temperature) and environmental data."

[0455] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0456] Step 1:

[0457] The terminal acquires biometric information such as heart rate and body temperature through the wearable device worn by the worker. This biometric information becomes the input data. The terminal collects this data using a sensor API and prepares to send it to the server.

[0458] Step 2:

[0459] The terminal acquires ambient environmental information using installed temperature and humidity sensors. This environmental information becomes the input data. The acquired temperature and humidity data is then organized for transmission to the server.

[0460] Step 3:

[0461] The terminal records sound wave information, particularly the worker's voice or ambient sounds, through a collection device. This sound wave data becomes the input data. The recorded audio data is converted to an audio format and prepared for transfer to the server.

[0462] Step 4:

[0463] The server integrates the received biometric, environmental, and acoustic information. Here, data in different formats is stored in a common database and prepared for the next analysis step. Data integrity is checked, and missing values ​​are imputed.

[0464] Step 5:

[0465] The server processes the integrated data using analytical tools. This analysis includes a stress detection algorithm. Based on the input data, it evaluates the worker's stress level and generates an output indicating high or low stress levels.

[0466] Step 6:

[0467] The server generates suggestions using a generation mechanism based on the analyzed results. In this step, recommendations for rest and exercise tailored to the stress level are specified and output as notification messages.

[0468] Step 7:

[0469] The user receives suggestions via a notification method. The suggestions appear on the device as pop-ups or voice messages. The user selects an action based on the recommendation and enters feedback on the device.

[0470] Step 8:

[0471] The server receives feedback from users. This feedback data is used for subsequent data analysis and improvement of the accuracy of suggestions. The automated learning model is updated and the changes are reflected in future analyses.

[0472] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0473] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0474] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0475] [Third Embodiment]

[0476] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0477] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0478] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0479] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0480] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0481] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0482] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0483] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0484] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0485] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0486] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0487] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0488] This invention is a support system for solving the problem of nighttime crying in infants during childcare, and includes a measuring device, a sensor device, a sound collection device, an analysis device, a generation device, and a notification device. This system has the function of collecting and integrating the infant's biological data, surrounding environment data, and crying sound data, and analyzing this data to identify the cause of nighttime crying and notify parents of appropriate countermeasures.

[0489] First, the device uses a wearable measuring device to acquire information about the baby's body temperature, heart rate, and movement. This information is usually acquired in real time and used to monitor the baby's health and stability. The device also acquires environmental information such as temperature and humidity using sensors installed in the room. This allows the device to constantly know whether the room is a comfortable environment for the baby.

[0490] The terminal's voice collection device records the baby's cries and generates audio data. This audio data is an important source of information for analyzing crying patterns. The obtained crying data is analyzed by a voice recognition algorithm, and the psychological and physiological state corresponding to the crying pattern is inferred.

[0491] Next, the server integrates this data and uses an analysis device to make a comprehensive assessment of the baby's condition. For example, even if the crying pattern indicates "hunger," it compares this data with other information such as body temperature and ambient temperature to determine the most appropriate course of action.

[0492] Based on the analysis results, the generator produces specific countermeasures. These countermeasures are presented in an intuitive and easy-to-understand way for parents, such as, "It is highly likely that the child is hungry, so we recommend breastfeeding."

[0493] The generated countermeasures are notified to the parent from the device via a notification device. The user can then take appropriate action immediately based on this notification. Furthermore, the parent's response is input into the system as feedback, which the server analyzes and uses its automatic learning function to improve the accuracy of future recommendations. This enables the continuous provision of personalized support.

[0494] The introduction of this system will allow parents to deal with their baby's nighttime crying problem with less stress and less time, and as a result, a significant improvement in the childcare environment is expected.

[0495] The following describes the processing flow.

[0496] Step 1:

[0497] The device uses a wearable measuring device to acquire biometric data such as the baby's body temperature, heart rate, and movement in real time.

[0498] Step 2:

[0499] The terminal uses sensor devices installed in the room to collect environmental data such as temperature, humidity, and lighting intensity.

[0500] Step 3:

[0501] The device uses a voice collection device to record the baby's cries and generate audio data.

[0502] Step 4:

[0503] The terminal sends all the data acquired in steps 1, 2, and 3 to the server via the communication module.

[0504] Step 5:

[0505] The server analyzes the received audio data, uses a speech recognition algorithm to identify crying patterns, and infers the state (for example, finding signs of hunger or discomfort).

[0506] Step 6:

[0507] The server integrates biometric and environmental data, and performs data analysis to assess the baby's overall condition, along with the results of the crying analysis.

[0508] Step 7:

[0509] Based on the analysis results, the server generates the optimal countermeasures for the identified causes via a generator.

[0510] Step 8:

[0511] The server sends the generated corrective actions to the terminal.

[0512] Step 9:

[0513] The device notifies the parent and provides recommended actions in real time (e.g., "The child may be hungry. Breastfeeding is recommended").

[0514] Step 10:

[0515] The user responds to the baby's needs based on the notification and inputs the results and feedback into the system via their device.

[0516] Step 11:

[0517] The server analyzes user feedback, updates its data analysis model using an automated learning algorithm, and aims to improve the accuracy of future countermeasure suggestions.

[0518] (Example 1)

[0519] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0520] A baby's nighttime crying is a major source of stress and burden for parents. Traditional methods often make it difficult to identify and address the cause of a baby's crying, and finding an appropriate solution is challenging. Therefore, there is a need for a system that can accurately identify the cause of a baby's crying and quickly implement appropriate countermeasures.

[0521] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0522] In this invention, the server includes measuring means for acquiring biological information, detection means for acquiring environmental information, and sound collection means for acquiring sound information. This enables comprehensive data analysis of the baby's physiological state and surrounding environment, allowing for the identification of the cause of the baby's crying and the rapid presentation of appropriate countermeasures to the parents.

[0523] "Biometric information" refers to data related to a baby's physical condition and reactions, such as body temperature, heart rate, and movement.

[0524] "Environmental information" refers to data related to the surrounding environmental conditions, such as temperature, humidity, and sound pressure in the space where the baby is located.

[0525] "Auditory information" refers to information expressed through sound, such as a baby crying.

[0526] "Measuring means" refers to devices and methods for acquiring biological information, which allows for monitoring the baby's health.

[0527] "Detection means" refers to devices and methods for sensing and acquiring environmental information, which allows us to obtain information to determine the comfort level of the environment in which a baby is present.

[0528] "Voice collection means" refers to devices and methods for acquiring voice information, which can be used to obtain data for analyzing patterns in a baby's cries.

[0529] "Analysis means" refers to devices and methods for integrating and analyzing acquired biological information, environmental information, and audio information to evaluate the baby's condition.

[0530] "Generating means" refers to devices or methods that create recommended countermeasures based on analyzed data.

[0531] "Notification means" refers to devices or methods for informing users, such as parents, of the generated countermeasures, thereby enabling a quick response.

[0532] This invention is a support system for resolving the problem of babies crying at night. This system consists of a server, terminals, and users, and acquires biometric information, environmental information, and voice information. By integrating and analyzing this data, it evaluates the baby's condition, generates appropriate countermeasures, and notifies the user.

[0533] The device acquires real-time biometric information such as the baby's body temperature, heart rate, and movement through a wearable measuring device. It also collects environmental information such as room temperature and humidity using environmental sensors. This allows for constant monitoring of whether the baby's living space is comfortable. Furthermore, a voice recording device records the baby's cries, acquiring audio information. All of this data is transmitted to a server via the network.

[0534] The server integrates the received data and uses advanced analytical equipment to comprehensively evaluate the baby's psychological and physiological state. This analysis utilizes a generative AI model, for example, analyzing the characteristics of crying to infer causes such as "hunger" or "discomfort." Based on the analysis results, the generator automatically produces specific countermeasures.

[0535] As a concrete example, consider a case where a baby starts crying in the middle of the night. The device immediately sends various data to the server. The server uses a generative AI model to determine that "the baby is likely hungry" and generates a notification recommending breastfeeding. An example of a prompt that the user can actually input at this time would be in the format of "The baby is crying. Body temperature 37 degrees, room temperature 26 degrees, heart rate 120, and hunger signs detected from audio data. What is the recommended course of action?"

[0536] The generated countermeasures are sent to the device via a notification device and quickly communicated to the parents. Users can immediately take action based on the received information, and by feeding the results back into the system, the server improves the accuracy of the next analysis through its automatic learning function. This entire system allows parents to deal with their baby's nighttime crying more efficiently.

[0537] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0538] Step 1:

[0539] The device uses a wearable measuring device to acquire real-time biometric data such as the baby's body temperature, heart rate, and movement. Through this input data, the system aims to identify the baby's physiological state and detect abnormalities early. The collected biometric information is then transmitted to a server as output.

[0540] Step 2:

[0541] The device uses environmental sensors to collect environmental information such as room temperature, humidity, and sound pressure. By acquiring this environmental data, it becomes possible to determine whether the space where the baby is present is comfortable and safe. The input environmental information is visualized through a dashboard and transferred to the server as output.

[0542] Step 3:

[0543] The device uses a voice collection device to record the baby's cries and acquire audio information. This audio data serves as foundational data for analyzing the baby's emotions and needs. The input crying audio data is processed digitally to extract features, which are then sent to the server as output.

[0544] Step 4:

[0545] The server integrates biometric, environmental, and audio data, and uses advanced analytical equipment to assess the baby's psychological and physiological state. This analysis uses a generative AI model to analyze the correlations between each input data point and identify the primary causes of nighttime crying. The output is a list of the baby's estimated needs.

[0546] Step 5:

[0547] Based on the analysis results, the server uses a generator to produce specific countermeasures. These are recommended actions based on the analyzed data, and are notified to the user in a way that allows for immediate action. For example, the output might be an instruction such as, "Breastfeeding is recommended due to hunger."

[0548] Step 6:

[0549] The notification device sends the generated countermeasures to the user's terminal. The user receives this notification and can take appropriate action quickly. Based on the generated instructions, the user takes action and feeds the results back to the system.

[0550] Step 7:

[0551] The server receives feedback from users and uses an automated learning function to improve the accuracy of subsequent analyses. This feedback updates the generated AI model, resulting in improved efficiency and accuracy at each processing step.

[0552] (Application Example 1)

[0553] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0554] In today's childcare environment, parents spend a great deal of stress and time dealing with their babies' nighttime crying. Ensuring safety within the home is also a significant concern. Managing home security while simultaneously monitoring the baby's health and environmental comfort in real time has been difficult with conventional methods. This invention provides a multi-functional management system to solve these problems.

[0555] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0556] In this invention, the server includes a measurement means for acquiring biometric information, a detection means for acquiring environmental information, and a sound information collection means for acquiring audio information. This makes it possible to grasp the baby's health condition and environmental changes in real time, identify the cause of nighttime crying, and improve safety in the home by detecting abnormal sounds and movements.

[0557] "Biometric information" refers to numerical data that indicates the baby's physical condition, such as body temperature, heart rate, and movement.

[0558] "Measurement means" refers to devices and technologies used to accurately acquire biological information.

[0559] "Environmental information" refers to data that indicates external conditions that affect a baby's comfort, such as room temperature, humidity, and lighting levels.

[0560] "Detection means" refers to devices and technologies for collecting environmental information.

[0561] "Audio information" refers to data related to sound, including sounds such as a baby crying or unusual noises in the home.

[0562] "Audio information acquisition means" refers to devices or technologies that record audio information and provide it in a format that allows for analysis.

[0563] "Analysis means" refers to devices and technologies that integrate acquired biological information, environmental information, and audio information to comprehensively assess the baby's condition and the home environment.

[0564] "Generating means" refers to devices or technologies that create appropriate countermeasures or warnings based on analysis results.

[0565] "Notification means" refers to devices or technologies used to inform parents or users of generated countermeasures or warnings.

[0566] "Security analysis means" refers to devices and technologies that analyze audio information within a home, detect anomalies, and generate warnings.

[0567] A "warning notification means" refers to a device or technology used to send a warning to the user regarding a detected anomaly.

[0568] To implement this invention, the server needs to be equipped with measurement means for acquiring biometric information, detection means for acquiring environmental information, and sound information collection means for acquiring audio information. Each means can use wearable devices or sensors installed in the home. Specifically, a wearable device attached to the baby's body measures body temperature, heart rate, and movement in real time. Environmental information is acquired by temperature and humidity sensors and lighting sensors installed in the room. Audio information is collected by a terminal equipped with a microphone. All of this information is transmitted to the server.

[0569] The server integrates and analyzes this acquired information using an analysis tool. The analysis tool is data analysis software that utilizes a generative AI model and identifies patterns in the baby's cries through speech recognition. Furthermore, it combines body temperature, heart rate, movement, environmental information, and crying data to determine the baby's condition. For example, if a pattern indicating hunger is detected from the crying, it is compared with the ambient room temperature to generate appropriate countermeasures.

[0570] The countermeasures and warnings generated during this process are created by the generation mechanism and then notified to the parent or user on the device via the notification mechanism. Push notifications using smartphones are used for notifications, and the baby's condition and recommended actions (e.g., "Change the diaper" or "Feed the baby milk") are displayed intuitively.

[0571] Furthermore, if the home security analysis system detects an unusual sound, a warning is generated and immediately sent to the user via a warning notification system. Upon receiving this notification, the user can quickly check the security of their home.

[0572] As a concrete example, a user could enter the prompt "How can I simultaneously detect a baby crying at night and other unusual noises in the home?" and the system could then provide information about detecting crying or other unusual noises. The introduction of this system would make it possible to reduce the burden of childcare while maintaining a safe home environment.

[0573] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0574] Step 1:

[0575] The measurement method uses a wearable device to measure the baby's body temperature, heart rate, and movement. The input is biometric data, which is sent to a server as digital data. This data is acquired in real time and used by the server to assess the baby's health.

[0576] Step 2:

[0577] The detection system collects environmental data within the home through sensors. The input includes environmental data such as temperature, humidity, and lighting levels, and this information is also sent to a server. This is necessary to analyze how the environment affects the baby's comfort.

[0578] Step 3:

[0579] The sound information collection system uses the device's microphone to collect sounds such as a baby crying and other household noises. The input is an audio signal, which is converted into a digital format and sent to the server. The audio data is used to identify patterns in nighttime crying and unusual sounds within the home.

[0580] Step 4:

[0581] The server integrates and analyzes this acquired data using analytical tools. The input is all the data obtained in steps 1 to 3. Using a generative AI model, it detects crying patterns and environmental changes to make a comprehensive judgment about the baby's condition and the home environment. The output is the condition judgment result.

[0582] Step 5:

[0583] The generation unit generates countermeasures and warnings based on the judgment results from the analysis unit. The input is the judgment result from step 4. For example, specific actions such as "change the diaper" or "feed the baby milk" are instructed. The output is the generated countermeasures and warnings.

[0584] Step 6:

[0585] The notification system informs the user of the generated countermeasures and warnings via the device. The input is the output from step 5. Information is delivered to the user intuitively via push notifications using a smartphone.

[0586] Step 7:

[0587] The security analysis system analyzes unusual sounds within the home and generates warnings as needed. The input is the audio data obtained in step 3. To ensure reliable security measures are taken, the server reacts immediately upon detecting an unusual sound. The output is the generated security warning.

[0588] Step 8:

[0589] The warning notification system notifies the user of security alerts. The input is the output of step 7. In cases of high urgency, a higher warning level is used to quickly draw the user's attention.

[0590] These steps make it possible to efficiently manage both the baby's health and home security during childcare.

[0591] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0592] This invention relates to a support system for reducing nighttime crying in babies and parental stress during childcare. The system includes a measuring device, a sensor device, a voice collection device, an analysis device, a generation device, a notification device, and an emotion engine, which enable the provision of coping strategies that take into account the user's emotional state.

[0593] First, the device acquires the baby's biometric information using a wearable measuring device. This is used to monitor body temperature, heart rate, movement, etc., in real time. In addition, the device collects temperature and humidity from installed sensor devices to obtain environmental information.

[0594] Next, the device records the baby's cries using a voice collection device and saves it as audio data. Based on this audio data, a voice recognition algorithm on the server analyzes the crying patterns and infers the baby's state, such as hunger, discomfort, or sleepiness.

[0595] In this invention, a newly introduced emotion engine on the server plays a major role. This emotion engine uses acquired voice data and parent biometric data to identify the parent's emotional state. For example, it can assess whether the parent is stressed or relaxed based on voice tone and biometric information (such as changes in heart rate).

[0596] The server integrates this data and uses an analysis device to comprehensively assess the state of the baby and parents. Based on this assessment, the generator takes the output of the emotion engine into account when generating coping strategies based on the identified causes. This allows for the recommendation of alternative approaches, for example, when parents are under stress.

[0597] The generated response measures are sent to the device via a notification system, and the parents are shown recommendations. These notifications include specific action plans, such as "Your baby may be hungry, please breastfeed them" or "Your stress levels are high, so we recommend taking a few minutes to breathe deeply."

[0598] Users can take appropriate action based on this notification and provide feedback on its effectiveness to the system. This feedback is sent to the server, and the system continuously improves through its automated learning function, enabling it to provide more accurate and personalized support.

[0599] This system allows parents to monitor their own emotional state while caring for their baby, reducing parenting stress and enabling them to parent more effectively.

[0600] The following describes the processing flow.

[0601] Step 1:

[0602] The device uses a wearable measuring device to acquire biometric data such as the baby's body temperature, heart rate, and movement in real time.

[0603] Step 2:

[0604] The terminal uses sensor devices installed in the room to collect environmental data such as temperature and humidity.

[0605] Step 3:

[0606] The device uses a voice collection device to record the baby's cries and generates audio data.

[0607] Step 4:

[0608] The terminal transmits all of the above acquired data to the server via the communication module.

[0609] Step 5:

[0610] The server analyzes the audio data and uses a speech recognition algorithm to identify crying patterns and infer the state of the crying.

[0611] Step 6:

[0612] The server integrates biometric and environmental data, and, along with the results of the crying analysis, assesses the baby's overall condition.

[0613] Step 7:

[0614] The server uses an emotion engine to analyze the user's voice tone and biometric data to identify the user's emotional state.

[0615] Step 8:

[0616] The server comprehensively evaluates the baby's and the user's condition and uses a generator to produce the optimal course of action. The user's emotional state is taken into consideration during this process.

[0617] Step 9:

[0618] The server sends the generated corrective actions to the terminal.

[0619] Step 10:

[0620] The device will notify the user of recommendations. These notifications will include specific advice on baby care and mental support for the user.

[0621] Step 11:

[0622] The user takes action based on the notification content and returns the results and feedback to the system via their device.

[0623] Step 12:

[0624] The server analyzes user feedback and uses automated learning algorithms to improve the system's recommendation accuracy and sentiment recognition accuracy.

[0625] (Example 2)

[0626] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0627] In childcare, nighttime crying of babies and parental stress are major challenges, and many parents experience anxiety and fatigue. This invention provides appropriate countermeasures based on the baby's biological information and crying, as well as the parent's biological information, thereby reducing the stress of childcare and enabling more effective childcare.

[0628] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0629] In this invention, the server includes measuring means for acquiring biological information, sensor means for acquiring environmental information, and voice acquisition means for acquiring voice information. This makes it possible to accurately evaluate the condition of the baby and the parent and to provide the parent with specific and useful countermeasures.

[0630] "Biometric information" refers to information obtained from the body of a baby or parent, such as body temperature, heart rate, and body movements.

[0631] "Environmental information" refers to information related to the environment of a given location, such as temperature and humidity.

[0632] "Audio information" refers to audio data that includes sounds such as a baby crying or other ambient sounds.

[0633] "Analysis means" refers to a processing mechanism that integrates acquired biological information, environmental information, and audio information to evaluate the state of the subject.

[0634] A "generating mechanism" is a mechanism that creates coping measures using an emotion evaluation mechanism based on the results of an analysis.

[0635] "Notification means" refers to a means of informing parents of the countermeasures generated by the generation means.

[0636] The "emotional evaluation mechanism" is a function that evaluates the emotional state of parents through auditory and biometric information.

[0637] The "automatic learning function" is a continuous learning function that uses user feedback to improve the accuracy of the system.

[0638] This invention is a system for parents to effectively manage their baby's nighttime crying and their own stress. This system primarily consists of three elements: a terminal, a server, and a user.

[0639] The device acquires the baby's biometric information through wearable measurement devices. Specifically, it uses devices such as smart bands and baby monitors to collect the baby's body temperature, heart rate, and movement in real time. It also uses sensor devices to collect environmental information such as room temperature and humidity.

[0640] The device uses a microphone-equipped voice collection device to record the baby's cries and saves them as audio data. This data is then sent to a server equipped with a data analysis algorithm.

[0641] The server uses speech recognition algorithms and emotion evaluation mechanisms to assess the state of the baby and parent. The audio data is analyzed using machine learning libraries such as TensorFlow and PyTorch to infer patterns in the baby's cries and the parent's emotional state.

[0642] The server's generation device generates appropriate countermeasures based on the evaluation results. It utilizes a generation AI model and is implemented in programming languages ​​such as Python. The generated countermeasures are sent to the device via a notification device, providing parents with specific action guidelines. For example, notifications such as "Your baby may be hungry, so we recommend breastfeeding" or "Your stress levels are high, so we recommend taking a few minutes to breathe deeply" may be sent.

[0643] Users can improve the efficiency of childcare and their own mental state by taking action based on notifications. User feedback is sent to the server and used by an automated learning function to improve the accuracy of the system.

[0644] As a concrete example of a prompt, one could ask the generating AI model, "What are some ways to reduce parental stress when a baby is crying?" In response to this inquiry, the system would suggest the most appropriate course of action.

[0645] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0646] Step 1:

[0647] The device acquires biometric information such as the baby's body temperature, heart rate, and movement using a wearable measuring device. The input is real-time sensor data obtained from the wearable device, which is then converted to a digital format for output. Specifically, biometric information is continuously transmitted when the smart band is worn.

[0648] Step 2:

[0649] The terminal uses sensor devices installed in the room to acquire environmental information such as temperature and humidity. The input is real-time data from the environmental sensors, which is then converted to a digital format and output. In operation, the sensor devices periodically measure the indoor environmental data and store it in a database.

[0650] Step 3:

[0651] The device records the baby's cries using a voice collection device. The input is an analog audio signal from a voice sensor, which is converted into digital audio data and output. Specifically, the microphone detects the baby's cries and saves them as an audio file.

[0652] Step 4:

[0653] The server analyzes the collected audio data using a speech recognition algorithm. The input is digital audio data, and it analyzes the audio patterns to generate output that estimates the cause of crying (hunger, discomfort, sleepiness, etc.). In terms of operation, it applies a model using machine learning libraries such as TensorFlow or PyTorch and analyzes the results.

[0654] Step 5:

[0655] The server identifies the parent's emotional state using voice data and the parent's biometric data. The input consists of changes in voice tone and biometric information, generating an output that evaluates emotional states such as stress and relaxation. Specifically, data analysis is performed through an emotion evaluation mechanism.

[0656] Step 6:

[0657] Based on the aforementioned analysis results, the server generates specific countermeasures through a generation device. The input is the assessment results of the baby's and parent's condition, and based on this, it outputs countermeasures including recommended actions. In operation, the optimal action plan is dynamically created using a generation AI model.

[0658] Step 7:

[0659] The notification device sends the generated corrective actions to the device and displays them to the parent. The input is corrective action guideline data, which is output as a user-friendly notification. Specifically, information such as "Your baby may be hungry, so we recommend breastfeeding" is visually displayed on the device screen.

[0660] Step 8:

[0661] Users take action based on the notified guidelines and provide feedback to the system regarding the results. The input consists of the user's actual actions and their outcomes, which the system receives and adds to its learning database as output. Through this feedback mechanism, the system continuously improves.

[0662] (Application Example 2)

[0663] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0664] In many workplaces, properly managing the physical and mental stress of workers is a challenge. Accumulated stress among workers can lead to decreased work efficiency and health problems, making real-time stress assessment and effective countermeasures necessary.

[0665] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0666] In this invention, the server includes measuring means for acquiring biological information, detection means for acquiring environmental information, and collection means for acquiring sound wave information. This makes it possible to monitor the physical and mental state of workers in real time and recommend appropriate measures.

[0667] "Biometric information" refers to data that indicates the physical condition of a worker, such as their heart rate and body temperature.

[0668] "Environmental information" refers to data that indicates external conditions of the work environment, such as temperature and humidity.

[0669] "Sound wave information" refers to data related to voice and other sounds, which is acquired to understand the conditions of workers and the environment.

[0670] "Measurement means" refers to devices and sensors used to acquire the biometric information of workers.

[0671] "Detection means" refers to devices or sensors used to acquire information about the work environment.

[0672] "Collection means" refers to devices or systems for acquiring sound wave information.

[0673] "Analysis means" refers to algorithms and systems that integrate acquired data to evaluate the worker's condition.

[0674] "Generation means" refers to a device or program that creates recommended countermeasures based on the analysis results.

[0675] A "notification means" is a device or system used to communicate the generated countermeasures to workers.

[0676] The system for realizing this application example operates through the collaboration of a server, a terminal, and a user. The server receives biometric information, environmental information, and sound wave information transmitted from the measurement means, detection means, and collection means. The measurement means acquires the worker's biometric information in real time using hardware such as heart rate sensors and body temperature sensors. The detection means acquires information about the work environment using temperature sensors and humidity sensors. The collection means acquires sound wave information using speech recognition technology.

[0677] The server integrates the acquired data and evaluates the worker's condition through analysis. This analysis uses speech recognition algorithms and machine learning models for stress detection. For example, the server might determine that a worker's stress level is increasing if their heart rate is higher than normal.

[0678] Based on the analysis results, the generation system proposes appropriate measures to the worker. These measures may include voice notifications or visual instructions via a display. For example, it might recommend that a worker experiencing high stress levels "take a 5-minute break."

[0679] Users manage their stress by accepting and implementing these suggestions via notification mechanisms. User feedback is sent to the server and used as feedback to an automated learning model to improve the accuracy of recommendations.

[0680] Examples of prompt statements to input into the generative AI model are as follows:

[0681] "Developer prompt: Please show us how to build an AI system that suggests measures to reduce worker stress in a factory environment. Please also provide example code that generates appropriate notifications based on biometric data (heart rate, body temperature) and environmental data."

[0682] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0683] Step 1:

[0684] The terminal acquires biometric information such as heart rate and body temperature through the wearable device worn by the worker. This biometric information becomes the input data. The terminal collects this data using a sensor API and prepares to send it to the server.

[0685] Step 2:

[0686] The terminal acquires ambient environmental information using installed temperature and humidity sensors. This environmental information becomes the input data. The acquired temperature and humidity data is then organized for transmission to the server.

[0687] Step 3:

[0688] The terminal records sound wave information, particularly the worker's voice or ambient sounds, through a collection device. This sound wave data becomes the input data. The recorded audio data is converted to an audio format and prepared for transfer to the server.

[0689] Step 4:

[0690] The server integrates the received biometric, environmental, and acoustic information. Here, data in different formats is stored in a common database and prepared for the next analysis step. Data integrity is checked, and missing values ​​are imputed.

[0691] Step 5:

[0692] The server processes the integrated data using analytical tools. This analysis includes a stress detection algorithm. Based on the input data, it evaluates the worker's stress level and generates an output indicating high or low stress levels.

[0693] Step 6:

[0694] The server generates suggestions using a generation mechanism based on the analyzed results. In this step, recommendations for rest and exercise tailored to the stress level are specified and output as notification messages.

[0695] Step 7:

[0696] The user receives suggestions via a notification method. The suggestions appear on the device as pop-ups or voice messages. The user selects an action based on the recommendation and enters feedback on the device.

[0697] Step 8:

[0698] The server receives feedback from users. This feedback data is used for subsequent data analysis and improvement of the accuracy of suggestions. The automated learning model is updated and the changes are reflected in future analyses.

[0699] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0700] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0701] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0702] [Fourth Embodiment]

[0703] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0704] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0705] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0706] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0707] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0708] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0709] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0710] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0711] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0712] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0713] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0714] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0715] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0716] This invention is a support system for solving the problem of nighttime crying in infants during childcare, and includes a measuring device, a sensor device, a sound collection device, an analysis device, a generation device, and a notification device. This system has the function of collecting and integrating the infant's biological data, surrounding environment data, and crying sound data, and analyzing this data to identify the cause of nighttime crying and notify parents of appropriate countermeasures.

[0717] First, the device uses a wearable measuring device to acquire information about the baby's body temperature, heart rate, and movement. This information is usually acquired in real time and used to monitor the baby's health and stability. The device also acquires environmental information such as temperature and humidity using sensors installed in the room. This allows the device to constantly know whether the room is a comfortable environment for the baby.

[0718] The terminal's voice collection device records the baby's cries and generates audio data. This audio data is an important source of information for analyzing crying patterns. The obtained crying data is analyzed by a voice recognition algorithm, and the psychological and physiological state corresponding to the crying pattern is inferred.

[0719] Next, the server integrates this data and uses an analysis device to make a comprehensive assessment of the baby's condition. For example, even if the crying pattern indicates "hunger," it compares this data with other information such as body temperature and ambient temperature to determine the most appropriate course of action.

[0720] Based on the analysis results, the generator produces specific countermeasures. These countermeasures are presented in an intuitive and easy-to-understand way for parents, such as, "It is highly likely that the child is hungry, so we recommend breastfeeding."

[0721] The generated countermeasures are notified to the parent from the device via a notification device. The user can then take appropriate action immediately based on this notification. Furthermore, the parent's response is input into the system as feedback, which the server analyzes and uses its automatic learning function to improve the accuracy of future recommendations. This enables the continuous provision of personalized support.

[0722] The introduction of this system will allow parents to deal with their baby's nighttime crying problem with less stress and less time, and as a result, a significant improvement in the childcare environment is expected.

[0723] The following describes the processing flow.

[0724] Step 1:

[0725] The device uses a wearable measuring device to acquire biometric data such as the baby's body temperature, heart rate, and movement in real time.

[0726] Step 2:

[0727] The terminal uses sensor devices installed in the room to collect environmental data such as temperature, humidity, and lighting intensity.

[0728] Step 3:

[0729] The device uses a voice collection device to record the baby's cries and generate audio data.

[0730] Step 4:

[0731] The terminal sends all the data acquired in steps 1, 2, and 3 to the server via the communication module.

[0732] Step 5:

[0733] The server analyzes the received audio data, uses a speech recognition algorithm to identify crying patterns, and infers the state (for example, finding signs of hunger or discomfort).

[0734] Step 6:

[0735] The server integrates biometric and environmental data, and performs data analysis to assess the baby's overall condition, along with the results of the crying analysis.

[0736] Step 7:

[0737] Based on the analysis results, the server generates the optimal countermeasures for the identified causes via a generator.

[0738] Step 8:

[0739] The server sends the generated corrective actions to the terminal.

[0740] Step 9:

[0741] The device notifies the parent and provides recommended actions in real time (e.g., "The child may be hungry. Breastfeeding is recommended").

[0742] Step 10:

[0743] The user responds to the baby's needs based on the notification and inputs the results and feedback into the system via their device.

[0744] Step 11:

[0745] The server analyzes user feedback, updates its data analysis model using an automated learning algorithm, and aims to improve the accuracy of future countermeasure suggestions.

[0746] (Example 1)

[0747] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0748] A baby's nighttime crying is a major source of stress and burden for parents. Traditional methods often make it difficult to identify and address the cause of a baby's crying, and finding an appropriate solution is challenging. Therefore, there is a need for a system that can accurately identify the cause of a baby's crying and quickly implement appropriate countermeasures.

[0749] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0750] In this invention, the server includes measuring means for acquiring biological information, detection means for acquiring environmental information, and sound collection means for acquiring sound information. This enables comprehensive data analysis of the baby's physiological state and surrounding environment, allowing for the identification of the cause of the baby's crying and the rapid presentation of appropriate countermeasures to the parents.

[0751] "Biometric information" refers to data related to a baby's physical condition and reactions, such as body temperature, heart rate, and movement.

[0752] "Environmental information" refers to data related to the surrounding environmental conditions, such as temperature, humidity, and sound pressure in the space where the baby is located.

[0753] "Auditory information" refers to information expressed through sound, such as a baby crying.

[0754] "Measuring means" refers to devices and methods for acquiring biological information, which allows for monitoring the baby's health.

[0755] "Detection means" refers to devices and methods for sensing and acquiring environmental information, which allows us to obtain information to determine the comfort level of the environment in which a baby is present.

[0756] "Voice collection means" refers to devices and methods for acquiring voice information, which can be used to obtain data for analyzing patterns in a baby's cries.

[0757] "Analysis means" refers to devices and methods for integrating and analyzing acquired biological information, environmental information, and audio information to evaluate the baby's condition.

[0758] "Generating means" refers to devices or methods that create recommended countermeasures based on analyzed data.

[0759] "Notification means" refers to devices or methods for informing users, such as parents, of the generated countermeasures, thereby enabling a quick response.

[0760] This invention is a support system for resolving the problem of babies crying at night. This system consists of a server, terminals, and users, and acquires biometric information, environmental information, and voice information. By integrating and analyzing this data, it evaluates the baby's condition, generates appropriate countermeasures, and notifies the user.

[0761] The device acquires real-time biometric information such as the baby's body temperature, heart rate, and movement through a wearable measuring device. It also collects environmental information such as room temperature and humidity using environmental sensors. This allows for constant monitoring of whether the baby's living space is comfortable. Furthermore, a voice recording device records the baby's cries, acquiring audio information. All of this data is transmitted to a server via the network.

[0762] The server integrates the received data and uses advanced analytical equipment to comprehensively evaluate the baby's psychological and physiological state. This analysis utilizes a generative AI model, for example, analyzing the characteristics of crying to infer causes such as "hunger" or "discomfort." Based on the analysis results, the generator automatically produces specific countermeasures.

[0763] As a concrete example, consider a case where a baby starts crying in the middle of the night. The device immediately sends various data to the server. The server uses a generative AI model to determine that "the baby is likely hungry" and generates a notification recommending breastfeeding. An example of a prompt that the user can actually input at this time would be in the format of "The baby is crying. Body temperature 37 degrees, room temperature 26 degrees, heart rate 120, and hunger signs detected from audio data. What is the recommended course of action?"

[0764] The generated countermeasures are sent to the device via a notification device and quickly communicated to the parents. Users can immediately take action based on the received information, and by feeding the results back into the system, the server improves the accuracy of the next analysis through its automatic learning function. This entire system allows parents to deal with their baby's nighttime crying more efficiently.

[0765] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0766] Step 1:

[0767] The device uses a wearable measuring device to acquire real-time biometric data such as the baby's body temperature, heart rate, and movement. Through this input data, the system aims to identify the baby's physiological state and detect abnormalities early. The collected biometric information is then transmitted to a server as output.

[0768] Step 2:

[0769] The device uses environmental sensors to collect environmental information such as room temperature, humidity, and sound pressure. By acquiring this environmental data, it becomes possible to determine whether the space where the baby is present is comfortable and safe. The input environmental information is visualized through a dashboard and transferred to the server as output.

[0770] Step 3:

[0771] The device uses a voice collection device to record the baby's cries and acquire audio information. This audio data serves as foundational data for analyzing the baby's emotions and needs. The input crying audio data is processed digitally to extract features, which are then sent to the server as output.

[0772] Step 4:

[0773] The server integrates biometric, environmental, and audio data, and uses advanced analytical equipment to assess the baby's psychological and physiological state. This analysis uses a generative AI model to analyze the correlations between each input data point and identify the primary causes of nighttime crying. The output is a list of the baby's estimated needs.

[0774] Step 5:

[0775] Based on the analysis results, the server uses a generator to produce specific countermeasures. These are recommended actions based on the analyzed data, and are notified to the user in a way that allows for immediate action. For example, the output might be an instruction such as, "Breastfeeding is recommended due to hunger."

[0776] Step 6:

[0777] The notification device sends the generated countermeasures to the user's terminal. The user receives this notification and can take appropriate action quickly. Based on the generated instructions, the user takes action and feeds the results back to the system.

[0778] Step 7:

[0779] The server receives feedback from users and uses an automated learning function to improve the accuracy of subsequent analyses. This feedback updates the generated AI model, resulting in improved efficiency and accuracy at each processing step.

[0780] (Application Example 1)

[0781] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0782] In today's childcare environment, parents spend a great deal of stress and time dealing with their babies' nighttime crying. Ensuring safety within the home is also a significant concern. Managing home security while simultaneously monitoring the baby's health and environmental comfort in real time has been difficult with conventional methods. This invention provides a multi-functional management system to solve these problems.

[0783] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0784] In this invention, the server includes a measurement means for acquiring biometric information, a detection means for acquiring environmental information, and a sound information collection means for acquiring audio information. This makes it possible to grasp the baby's health condition and environmental changes in real time, identify the cause of nighttime crying, and improve safety in the home by detecting abnormal sounds and movements.

[0785] "Biometric information" refers to numerical data that indicates the baby's physical condition, such as body temperature, heart rate, and movement.

[0786] "Measurement means" refers to devices and technologies used to accurately acquire biological information.

[0787] "Environmental information" refers to data that indicates external conditions that affect a baby's comfort, such as room temperature, humidity, and lighting levels.

[0788] "Detection means" refers to devices and technologies for collecting environmental information.

[0789] "Audio information" refers to data related to sound, including sounds such as a baby crying or unusual noises in the home.

[0790] "Audio information acquisition means" refers to devices or technologies that record audio information and provide it in a format that allows for analysis.

[0791] "Analysis means" refers to devices and technologies that integrate acquired biological information, environmental information, and audio information to comprehensively assess the baby's condition and the home environment.

[0792] "Generating means" refers to devices or technologies that create appropriate countermeasures or warnings based on analysis results.

[0793] "Notification means" refers to devices or technologies used to inform parents or users of generated countermeasures or warnings.

[0794] "Security analysis means" refers to devices and technologies that analyze audio information within a home, detect anomalies, and generate warnings.

[0795] A "warning notification means" refers to a device or technology used to send a warning to the user regarding a detected anomaly.

[0796] To implement this invention, the server needs to be equipped with measurement means for acquiring biometric information, detection means for acquiring environmental information, and sound information collection means for acquiring audio information. Each means can use wearable devices or sensors installed in the home. Specifically, a wearable device attached to the baby's body measures body temperature, heart rate, and movement in real time. Environmental information is acquired by temperature and humidity sensors and lighting sensors installed in the room. Audio information is collected by a terminal equipped with a microphone. All of this information is transmitted to the server.

[0797] The server integrates and analyzes this acquired information using an analysis tool. The analysis tool is data analysis software that utilizes a generative AI model and identifies patterns in the baby's cries through speech recognition. Furthermore, it combines body temperature, heart rate, movement, environmental information, and crying data to determine the baby's condition. For example, if a pattern indicating hunger is detected from the crying, it is compared with the ambient room temperature to generate appropriate countermeasures.

[0798] The countermeasures and warnings generated during this process are created by the generation mechanism and then notified to the parent or user on the device via the notification mechanism. Push notifications using smartphones are used for notifications, and the baby's condition and recommended actions (e.g., "Change the diaper" or "Feed the baby milk") are displayed intuitively.

[0799] Furthermore, if the home security analysis system detects an unusual sound, a warning is generated and immediately sent to the user via a warning notification system. Upon receiving this notification, the user can quickly check the security of their home.

[0800] As a concrete example, a user could enter the prompt "How can I simultaneously detect a baby crying at night and other unusual noises in the home?" and the system could then provide information about detecting crying or other unusual noises. The introduction of this system would make it possible to reduce the burden of childcare while maintaining a safe home environment.

[0801] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0802] Step 1:

[0803] The measurement method uses a wearable device to measure the baby's body temperature, heart rate, and movement. The input is biometric data, which is sent to a server as digital data. This data is acquired in real time and used by the server to assess the baby's health.

[0804] Step 2:

[0805] The detection system collects environmental data within the home through sensors. The input includes environmental data such as temperature, humidity, and lighting levels, and this information is also sent to a server. This is necessary to analyze how the environment affects the baby's comfort.

[0806] Step 3:

[0807] The sound information collection system uses the device's microphone to collect sounds such as a baby crying and other household noises. The input is an audio signal, which is converted into a digital format and sent to the server. The audio data is used to identify patterns in nighttime crying and unusual sounds within the home.

[0808] Step 4:

[0809] The server integrates and analyzes this acquired data using analytical tools. The input is all the data obtained in steps 1 to 3. Using a generative AI model, it detects crying patterns and environmental changes to make a comprehensive judgment about the baby's condition and the home environment. The output is the condition judgment result.

[0810] Step 5:

[0811] The generation unit generates countermeasures and warnings based on the judgment results from the analysis unit. The input is the judgment result from step 4. For example, specific actions such as "change the diaper" or "feed the baby milk" are instructed. The output is the generated countermeasures and warnings.

[0812] Step 6:

[0813] The notification system informs the user of the generated countermeasures and warnings via the device. The input is the output from step 5. Information is delivered to the user intuitively via push notifications using a smartphone.

[0814] Step 7:

[0815] The security analysis system analyzes unusual sounds within the home and generates warnings as needed. The input is the audio data obtained in step 3. To ensure reliable security measures are taken, the server reacts immediately upon detecting an unusual sound. The output is the generated security warning.

[0816] Step 8:

[0817] The warning notification system notifies the user of security alerts. The input is the output of step 7. In cases of high urgency, a higher warning level is used to quickly draw the user's attention.

[0818] These steps make it possible to efficiently manage both the baby's health and home security during childcare.

[0819] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0820] This invention relates to a support system for reducing nighttime crying in babies and parental stress during childcare. The system includes a measuring device, a sensor device, a voice collection device, an analysis device, a generation device, a notification device, and an emotion engine, which enable the provision of coping strategies that take into account the user's emotional state.

[0821] First, the device acquires the baby's biometric information using a wearable measuring device. This is used to monitor body temperature, heart rate, movement, etc., in real time. In addition, the device collects temperature and humidity from installed sensor devices to obtain environmental information.

[0822] Next, the device records the baby's cries using a voice collection device and saves it as audio data. Based on this audio data, a voice recognition algorithm on the server analyzes the crying patterns and infers the baby's state, such as hunger, discomfort, or sleepiness.

[0823] In this invention, a newly introduced emotion engine on the server plays a major role. This emotion engine uses acquired voice data and parent biometric data to identify the parent's emotional state. For example, it can assess whether the parent is stressed or relaxed based on voice tone and biometric information (such as changes in heart rate).

[0824] The server integrates this data and uses an analysis device to comprehensively assess the state of the baby and parents. Based on this assessment, the generator takes the output of the emotion engine into account when generating coping strategies based on the identified causes. This allows for the recommendation of alternative approaches, for example, when parents are under stress.

[0825] The generated response measures are sent to the device via a notification system, and the parents are shown recommendations. These notifications include specific action plans, such as "Your baby may be hungry, please breastfeed them" or "Your stress levels are high, so we recommend taking a few minutes to breathe deeply."

[0826] Users can take appropriate action based on this notification and provide feedback on its effectiveness to the system. This feedback is sent to the server, and the system continuously improves through its automated learning function, enabling it to provide more accurate and personalized support.

[0827] This system allows parents to monitor their own emotional state while caring for their baby, reducing parenting stress and enabling them to parent more effectively.

[0828] The following describes the processing flow.

[0829] Step 1:

[0830] The device uses a wearable measuring device to acquire biometric data such as the baby's body temperature, heart rate, and movement in real time.

[0831] Step 2:

[0832] The terminal uses sensor devices installed in the room to collect environmental data such as temperature and humidity.

[0833] Step 3:

[0834] The device uses a voice collection device to record the baby's cries and generates audio data.

[0835] Step 4:

[0836] The terminal transmits all of the above acquired data to the server via the communication module.

[0837] Step 5:

[0838] The server analyzes the audio data and uses a speech recognition algorithm to identify crying patterns and infer the state of the crying.

[0839] Step 6:

[0840] The server integrates biometric and environmental data, and, along with the results of the crying analysis, assesses the baby's overall condition.

[0841] Step 7:

[0842] The server uses an emotion engine to analyze the user's voice tone and biometric data to identify the user's emotional state.

[0843] Step 8:

[0844] The server comprehensively evaluates the baby's and the user's condition and uses a generator to produce the optimal course of action. The user's emotional state is taken into consideration during this process.

[0845] Step 9:

[0846] The server sends the generated corrective actions to the terminal.

[0847] Step 10:

[0848] The device will notify the user of recommendations. These notifications will include specific advice on baby care and mental support for the user.

[0849] Step 11:

[0850] The user takes action based on the notification content and returns the results and feedback to the system via their device.

[0851] Step 12:

[0852] The server analyzes user feedback and uses automated learning algorithms to improve the system's recommendation accuracy and sentiment recognition accuracy.

[0853] (Example 2)

[0854] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0855] In childcare, nighttime crying of babies and parental stress are major challenges, and many parents experience anxiety and fatigue. This invention provides appropriate countermeasures based on the baby's biological information and crying, as well as the parent's biological information, thereby reducing the stress of childcare and enabling more effective childcare.

[0856] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0857] In this invention, the server includes measuring means for acquiring biological information, sensor means for acquiring environmental information, and voice acquisition means for acquiring voice information. This makes it possible to accurately evaluate the condition of the baby and the parent and to provide the parent with specific and useful countermeasures.

[0858] "Biometric information" refers to information obtained from the body of a baby or parent, such as body temperature, heart rate, and body movements.

[0859] "Environmental information" refers to information related to the environment of a given location, such as temperature and humidity.

[0860] "Audio information" refers to audio data that includes sounds such as a baby crying or other ambient sounds.

[0861] "Analysis means" refers to a processing mechanism that integrates acquired biological information, environmental information, and audio information to evaluate the state of the subject.

[0862] A "generating mechanism" is a mechanism that creates coping measures using an emotion evaluation mechanism based on the results of an analysis.

[0863] "Notification means" refers to a means of informing parents of the countermeasures generated by the generation means.

[0864] The "emotional evaluation mechanism" is a function that evaluates the emotional state of parents through auditory and biometric information.

[0865] The "automatic learning function" is a continuous learning function that uses user feedback to improve the accuracy of the system.

[0866] This invention is a system for parents to effectively manage their baby's nighttime crying and their own stress. This system primarily consists of three elements: a terminal, a server, and a user.

[0867] The device acquires the baby's biometric information through wearable measurement devices. Specifically, it uses devices such as smart bands and baby monitors to collect the baby's body temperature, heart rate, and movement in real time. It also uses sensor devices to collect environmental information such as room temperature and humidity.

[0868] The device uses a microphone-equipped voice collection device to record the baby's cries and saves them as audio data. This data is then sent to a server equipped with a data analysis algorithm.

[0869] The server uses speech recognition algorithms and emotion evaluation mechanisms to assess the state of the baby and parent. The audio data is analyzed using machine learning libraries such as TensorFlow and PyTorch to infer patterns in the baby's cries and the parent's emotional state.

[0870] The server's generation device generates appropriate countermeasures based on the evaluation results. It utilizes a generation AI model and is implemented in programming languages ​​such as Python. The generated countermeasures are sent to the device via a notification device, providing parents with specific action guidelines. For example, notifications such as "Your baby may be hungry, so we recommend breastfeeding" or "Your stress levels are high, so we recommend taking a few minutes to breathe deeply" may be sent.

[0871] Users can improve the efficiency of childcare and their own mental state by taking action based on notifications. User feedback is sent to the server and used by an automated learning function to improve the accuracy of the system.

[0872] As a concrete example of a prompt, one could ask the generating AI model, "What are some ways to reduce parental stress when a baby is crying?" In response to this inquiry, the system would suggest the most appropriate course of action.

[0873] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0874] Step 1:

[0875] The device acquires biometric information such as the baby's body temperature, heart rate, and movement using a wearable measuring device. The input is real-time sensor data obtained from the wearable device, which is then converted to a digital format for output. Specifically, biometric information is continuously transmitted when the smart band is worn.

[0876] Step 2:

[0877] The terminal uses sensor devices installed in the room to acquire environmental information such as temperature and humidity. The input is real-time data from the environmental sensors, which is then converted to a digital format and output. In operation, the sensor devices periodically measure the indoor environmental data and store it in a database.

[0878] Step 3:

[0879] The device records the baby's cries using a voice collection device. The input is an analog audio signal from a voice sensor, which is converted into digital audio data and output. Specifically, the microphone detects the baby's cries and saves them as an audio file.

[0880] Step 4:

[0881] The server analyzes the collected audio data using a speech recognition algorithm. The input is digital audio data, and it analyzes the audio patterns to generate output that estimates the cause of crying (hunger, discomfort, sleepiness, etc.). In terms of operation, it applies a model using machine learning libraries such as TensorFlow or PyTorch and analyzes the results.

[0882] Step 5:

[0883] The server identifies the parent's emotional state using voice data and the parent's biometric data. The input consists of changes in voice tone and biometric information, generating an output that evaluates emotional states such as stress and relaxation. Specifically, data analysis is performed through an emotion evaluation mechanism.

[0884] Step 6:

[0885] Based on the aforementioned analysis results, the server generates specific countermeasures through a generation device. The input is the assessment results of the baby's and parent's condition, and based on this, it outputs countermeasures including recommended actions. In operation, the optimal action plan is dynamically created using a generation AI model.

[0886] Step 7:

[0887] The notification device sends the generated corrective actions to the device and displays them to the parent. The input is corrective action guideline data, which is output as a user-friendly notification. Specifically, information such as "Your baby may be hungry, so we recommend breastfeeding" is visually displayed on the device screen.

[0888] Step 8:

[0889] Users take action based on the notified guidelines and provide feedback to the system regarding the results. The input consists of the user's actual actions and their outcomes, which the system receives and adds to its learning database as output. Through this feedback mechanism, the system continuously improves.

[0890] (Application Example 2)

[0891] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0892] In many workplaces, properly managing the physical and mental stress of workers is a challenge. Accumulated stress among workers can lead to decreased work efficiency and health problems, making real-time stress assessment and effective countermeasures necessary.

[0893] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0894] In this invention, the server includes measuring means for acquiring biological information, detection means for acquiring environmental information, and collection means for acquiring sound wave information. This makes it possible to monitor the physical and mental state of workers in real time and recommend appropriate measures.

[0895] "Biometric information" refers to data that indicates the physical condition of a worker, such as their heart rate and body temperature.

[0896] "Environmental information" refers to data that indicates external conditions of the work environment, such as temperature and humidity.

[0897] "Sound wave information" refers to data related to voice and other sounds, which is acquired to understand the conditions of workers and the environment.

[0898] "Measurement means" refers to devices and sensors used to acquire the biometric information of workers.

[0899] "Detection means" refers to devices or sensors used to acquire information about the work environment.

[0900] "Collection means" refers to devices or systems for acquiring sound wave information.

[0901] "Analysis means" refers to algorithms and systems that integrate acquired data to evaluate the worker's condition.

[0902] "Generation means" refers to a device or program that creates recommended countermeasures based on the analysis results.

[0903] A "notification means" is a device or system used to communicate the generated countermeasures to workers.

[0904] The system for realizing this application example operates through the collaboration of a server, a terminal, and a user. The server receives biometric information, environmental information, and sound wave information transmitted from the measurement means, detection means, and collection means. The measurement means acquires the worker's biometric information in real time using hardware such as heart rate sensors and body temperature sensors. The detection means acquires information about the work environment using temperature sensors and humidity sensors. The collection means acquires sound wave information using speech recognition technology.

[0905] The server integrates the acquired data and evaluates the worker's condition through analysis. This analysis uses speech recognition algorithms and machine learning models for stress detection. For example, the server might determine that a worker's stress level is increasing if their heart rate is higher than normal.

[0906] Based on the analysis results, the generation system proposes appropriate measures to the worker. These measures may include voice notifications or visual instructions via a display. For example, it might recommend that a worker experiencing high stress levels "take a 5-minute break."

[0907] Users manage their stress by accepting and implementing these suggestions via notification mechanisms. User feedback is sent to the server and used as feedback to an automated learning model to improve the accuracy of recommendations.

[0908] Examples of prompt statements to input into the generative AI model are as follows:

[0909] "Developer prompt: Please show us how to build an AI system that suggests measures to reduce worker stress in a factory environment. Please also provide example code that generates appropriate notifications based on biometric data (heart rate, body temperature) and environmental data."

[0910] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0911] Step 1:

[0912] The terminal acquires biometric information such as heart rate and body temperature through the wearable device worn by the worker. This biometric information becomes the input data. The terminal collects this data using a sensor API and prepares to send it to the server.

[0913] Step 2:

[0914] The terminal acquires ambient environmental information using installed temperature and humidity sensors. This environmental information becomes the input data. The acquired temperature and humidity data is then organized for transmission to the server.

[0915] Step 3:

[0916] The terminal records sound wave information, particularly the worker's voice or ambient sounds, through a collection device. This sound wave data becomes the input data. The recorded audio data is converted to an audio format and prepared for transfer to the server.

[0917] Step 4:

[0918] The server integrates the received biometric, environmental, and acoustic information. Here, data in different formats is stored in a common database and prepared for the next analysis step. Data integrity is checked, and missing values ​​are imputed.

[0919] Step 5:

[0920] The server processes the integrated data using analytical tools. This analysis includes a stress detection algorithm. Based on the input data, it evaluates the worker's stress level and generates an output indicating high or low stress levels.

[0921] Step 6:

[0922] The server generates suggestions using a generation mechanism based on the analyzed results. In this step, recommendations for rest and exercise tailored to the stress level are specified and output as notification messages.

[0923] Step 7:

[0924] The user receives suggestions via a notification method. The suggestions appear on the device as pop-ups or voice messages. The user selects an action based on the recommendation and enters feedback on the device.

[0925] Step 8:

[0926] The server receives feedback from users. This feedback data is used for subsequent data analysis and improvement of the accuracy of suggestions. The automated learning model is updated and the changes are reflected in future analyses.

[0927] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0928] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0929] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0930] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0931] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0932] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0933] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0934] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0935] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0936] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0937] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0938] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0939] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0940] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0941] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0942] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0943] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0944] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0945] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0946] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0947] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0948] The following is further disclosed regarding the embodiments described above.

[0949] (Claim 1)

[0950] A measuring device for acquiring biological data,

[0951] Sensor devices for acquiring environmental data,

[0952] A voice acquisition device for acquiring voice data,

[0953] An analytical device that integrates and analyzes the acquired data and evaluates the state of the target,

[0954] A generating device that generates recommended countermeasures based on the aforementioned analysis results,

[0955] A notification device that notifies the generated countermeasures,

[0956] A system that includes this.

[0957] (Claim 2)

[0958] The system according to claim 1, characterized in that the analysis device has a function to automatically learn the state of the target based on data analysis.

[0959] (Claim 3)

[0960] The system according to claim 1, characterized in that the notification device receives feedback from a user, and the generation device uses that feedback to improve the recommendation accuracy.

[0961] "Example 1"

[0962] (Claim 1)

[0963] Measurement means for acquiring biological information,

[0964] A detection means for acquiring environmental information,

[0965] A means for acquiring audio information,

[0966] An analysis means for integrating and analyzing the acquired information and determining the state of the target,

[0967] A generation means for generating recommended countermeasures based on the analysis results,

[0968] A notification means for notifying the generated countermeasures,

[0969] A system that includes this.

[0970] (Claim 2)

[0971] The system according to claim 1, characterized in that the analysis means has a function to automatically learn the state of the target based on information analysis.

[0972] (Claim 3)

[0973] The system according to claim 1, characterized in that the notification means receives feedback from the user, and the generation means uses that feedback to improve the recommendation accuracy.

[0974] "Application Example 1"

[0975] (Claim 1)

[0976] Measurement means for acquiring biological information,

[0977] A detection means for acquiring environmental information,

[0978] A means for collecting sound information to acquire audio information,

[0979] An analysis means for integrating and analyzing the acquired information and evaluating the state of the object to be evaluated,

[0980] A generation means for generating recommended countermeasures based on the analysis results,

[0981] A notification means for notifying the generated countermeasures,

[0982] A security analysis tool that analyzes abnormal information within the home and generates warnings,

[0983] A warning notification means for sending the aforementioned warning to the user,

[0984] A system that includes this.

[0985] (Claim 2)

[0986] The system according to claim 1, characterized in that the analysis means has a function to automatically learn the state of the object to be evaluated based on information analysis.

[0987] (Claim 3)

[0988] The system according to claim 1, characterized in that the notification means receives feedback from a user, and the generation means uses that feedback to improve recommendation accuracy.

[0989] "Example 2 of combining an emotion engine"

[0990] (Claim 1)

[0991] Measurement means for acquiring biological information,

[0992] Sensor means for acquiring environmental information,

[0993] A means for acquiring audio information,

[0994] An analytical means for integrating the acquired information and evaluating the condition of the baby and parent,

[0995] Based on the analysis results, a generation means for generating coping measures using an emotion evaluation mechanism,

[0996] A notification means for notifying the parent of the generated countermeasures,

[0997] A system that includes this.

[0998] (Claim 2)

[0999] The system according to claim 1, characterized in that the analysis means includes an emotion evaluation mechanism that evaluates the parent's emotional state using voice information and biometric information, and has an automatic learning function.

[1000] (Claim 3)

[1001] The system according to claim 1, characterized in that the notification means receives feedback from a user, and the generation means uses that feedback to improve the accuracy of recommending countermeasures.

[1002] "Application example 2 of combining emotional engines"

[1003] (Claim 1)

[1004] Measurement means for acquiring biological information,

[1005] A detection means for acquiring environmental information,

[1006] A means for acquiring sound wave information,

[1007] An analysis means for integrating and analyzing the acquired data and evaluating the worker's condition,

[1008] A generation means for generating recommended countermeasures based on the aforementioned analysis results,

[1009] A notification means for transmitting the generated countermeasures,

[1010] A system that includes this.

[1011] (Claim 2)

[1012] The system according to claim 1, characterized in that the analysis means has a function to automatically learn the worker's state based on information analysis.

[1013] (Claim 3)

[1014] The system according to claim 1, characterized in that the notification means receives a response from the operator, and the generation means uses the response to improve the accuracy of the proposal. [Explanation of symbols]

[1015] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A measuring device for acquiring biological data, Sensor devices for acquiring environmental data, A voice acquisition device for acquiring voice data, An analytical device that integrates and analyzes the acquired data and evaluates the state of the target, A generating device that generates recommended countermeasures based on the aforementioned analysis results, A notification device that notifies the generated countermeasures, A system that includes this.

2. The system according to claim 1, characterized in that the analysis device has a function to automatically learn the state of the target based on data analysis.

3. The system according to claim 1, characterized in that the notification device receives feedback from a user, and the generation device uses that feedback to improve the recommendation accuracy.

Images