Control signal generation device, control signal generation method, control signal generation program, and ayashi provision system

Abstract

It can provide a wide variety of soothing actions to infants and toddlers, effectively calming them down. [Solution] A control signal generating device that generates a control signal to be output to an output device that performs soothing actions on a target person, and comprises a controller. The controller detects the target person's unhappy state, creates a prompt to cause a generating AI to generate information on soothing actions corresponding to the detected unhappy state, outputs the created prompt to the generating AI, obtains a response to the prompt from the generating AI, and generates the control signal based on the obtained response.

Description

Technical Field

[0001] The present invention relates to a control signal generation device, a control signal generation method, a control signal generation program, and a coddling providing system.

Background Art

[0002] Conventionally, techniques related to coddling operations for infants, which respond to the crying sounds of infants lying on a bed or the like and perform responses such as coddling the infants, are known. For example, a technique has been proposed in which voice, video, and movement of an infant are acquired as data, and sound data corresponding to the comparison result between the data and comparison source data previously stored is emitted from a speaker to coddle and care for the infant (see, for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in the case of the conventional technology, sound data (coddling operation) is acquired using comparison source data stored in advance, and the coddling operation is a repetition of the same thing without change, resulting in a monotonous response. As a result, there is a concern that the infant may become bored with the coddling operation, the effect of coddling may decrease, and it may not be possible to appropriately coddle the infant, and there is a risk that the infant's irritated state cannot be resolved.

[0005] In view of the above problems, an object of the present invention is to provide a technology capable of providing various coddling operations to an infant and effectively coddling the infant.

Means for Solving the Problems

[0006] An exemplary control signal generation device of the present invention is a control signal generation device that generates a control signal to be output to an output device that performs soothing actions on a target person, and comprises a controller. The controller detects the target person's unhappy state, creates a prompt for a generative AI to generate information on soothing actions corresponding to the detected unhappy state, outputs the created prompt to the generative AI (Generative Artificial Intelligence), obtains a response to the prompt from the generative AI, and generates the control signal based on the obtained response. [Effects of the Invention]

[0007] According to the present invention, when generating control signals to output devices that perform soothing actions (e.g., display devices, speakers, vibration devices), prompts containing information that do not require user input are automatically created and input to a generation AI, and control signals are generated based on the AI's response. Furthermore, since the control signals for soothing actions are not generated based on pre-prepared information, a wide variety of soothing actions can be provided to infants. Therefore, it becomes possible to effectively soothe infants and appropriately resolve their fussy state. In addition, even when soothing actions are needed when the infant's guardian or other caregiver is unable to attend to the infant, the amount of user input required is reduced, so soothing actions can be performed automatically and with little difficulty. [Brief explanation of the drawing]

[0008] [Figure 1] Overall configuration diagram of the amusement provision system of this embodiment [Figure 2] Block diagram showing the configuration of the ayashi provision system in Figure 1. [Figure 3] A diagram showing an example of a pre-information table. [Figure 4] This diagram shows a two-dimensional model (psychological plane), which is an example of an emotion estimation model. [Figure 5] A diagram showing an example of a data table for the cause of an incident. [Figure 6]A diagram showing an example of a soothing behavior data table. [Figure 7] A flowchart illustrating the control signal generation process performed by the control signal generation device. [Modes for carrying out the invention]

[0009] Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the drawings. However, the present invention is not limited to the embodiments described below.

[0010] <1. Suspiciousness Provision System> Figure 1 is an overall configuration diagram of the soothing system 1 of this embodiment. The soothing system 1 is a system that performs soothing actions on a target person T1. In this embodiment, the soothing system 1 is intended to soothe an infant or toddler riding in vehicle Ca1, and to support the driver D1, who is a parent or guardian, by soothing the unhappy target person T1, thereby reducing the effort required for the driver D1 to soothe the infant or toddler.

[0011] The "Suspiciousness Provision System 1" includes a control signal generation device 10, a camera 2, a microphone 3, a display device 4, a speaker 5, and a vibration device 6, all mounted on the vehicle Ca1 (see Figures 1 and 2). The "Suspiciousness Provision System 1" also includes a generation AI device 20, which is a server device, as an additional device besides those mounted on the vehicle Ca1. These vehicle Ca1-mounted devices and the generation AI device 20 (server device) are connected via a wireless communication network N to constitute the "Suspiciousness Provision System 1". The control signal generation device 10 may, for example, be included in a drive recorder or navigation system installed in the vehicle Ca1.

[0012] The control signal generator 10 is a device that generates control signals to output devices that perform soothing actions on the target person T1. In this embodiment, the target person T1 is an infant riding in the back seat of vehicle Ca1 (the vehicle itself). The control signal generator 10 detects the unhappy state of the target person T1 (infant) and outputs control signals to various actuators (output devices) that act on the target person T1 (stimulating the five senses such as sight, hearing, and touch) in order to perform soothing actions that correspond to (alleviate) the unhappy state. These various actuators (output devices) include a display device 4, a speaker 5, and a vibration device 6, and the control signals are image signals and sound signals.

[0013] Camera 2, microphone 3, display device 4, speaker 5, and vibration device 6 are installed in vehicle Ca1. Camera 2 acquires (captures) external information (images) of the target person T1 of the soothing action. Microphone 3 acquires the voice of the target person T1. Display device 4 displays images and videos related to the soothing action. Speaker 5 outputs sounds related to the soothing action. Vibration device 6 includes an exciter composed of coils and magnets, a piezoelectric element, etc., and vibrates in response to the input signal to vibrate the target person T1. In other words, the display device 4, speaker 5, and vibration device 6 correspond to the various actuators that act on the target person T1 as described above. In addition, camera 2 and microphone 3 correspond to state detection sensors that detect the state of the target person T1.

[0014] The various actuators, including the display device 4, speaker 5, and vibration device 6, are output devices that perform soothing actions on the subject T1. More specifically, the display device 4, speaker 5, and vibration device 6 acquire control signals (image signals, audio signals, and vibration signals) related to soothing actions from the control signal generation device 10, and perform soothing actions (image display, audio output, and vibration output) on the subject T1 in accordance with the acquired control signals. The display device 4 displays images and videos related to the soothing actions. The speaker 5 outputs audio related to the soothing actions.

[0015] Note that the camera 2, the microphone 3, the display device 4, and the speaker 5 may be components included in, for example, a drive recorder or a navigation device provided in the vehicle Ca1.

[0016] FIG. 2 is a block diagram showing the configuration of the suspiciousness providing system 1 in FIG. 1. In FIG. 2, components necessary for explaining the features of the present embodiment are shown, and descriptions of general components are omitted.

[0017] <2. Control Signal Generation Device> The control signal generation device 10 includes a communication unit 11, a storage unit 12, and a controller 13.

[0018] The communication unit 11 is an interface for performing data communication with other devices (external generation AI device 20 (server device), camera 2, etc.) via a communication network. The communication unit 11 includes a communication device for performing wired communication and wireless communication with other devices. As the wireless communication device, for communication with the external generation AI device 20, for example, it is composed of a transceiver of a mobile phone network of 5G communication (5th generation mobile communication system). Also, for communication with each mounted device in the vehicle Ca1 such as the camera 2, for example, it is composed of a transceiver such as CAN (Controller Area Network), WiFi (Wireless Fidelity, registered trademark).

[0019] The storage unit 12 is configured to include a volatile memory and a non-volatile memory, and stores various information necessary for content playback processing. The volatile memory is composed of, for example, RAM (Random Access Memory). The non-volatile memory is composed of, for example, ROM (Read Only Memory), flash memory, and hard disk drive. Programs and data readable by the controller 13 are stored in the non-volatile memory. At least a part of the programs and data stored in the non-volatile memory may be configured to be acquired from other computer devices connected by wire or wirelessly, or from a portable recording medium.

[0020] The memory unit 12 stores a control signal generation program 121, a pre-information table 122, a prompt template table 123, a cause data table 124, and a soothing operation data table 125. The contents of these programs, data tables, etc., stored in the memory unit 12 are described separately. Furthermore, the memory unit 12 stores multiple data tables, etc., for various processing.

[0021] The controller 13 consists of a processor that performs calculations and other processing, and controls various operations in the control signal generation device 10. The processor includes, for example, a CPU (Central Processing Unit). The controller 13 executes the control signal generation program 121 stored in the memory unit 12 to realize various functions of the control signal generation device 10.

[0022] The controller 13 includes, as its functions, a target information acquisition unit 131, a state detection unit 132, a cause analysis unit 133, a corresponding operation acquisition unit 134, a detailed operation acquisition unit 135, a control signal generation unit 136, and a provision unit 137. In this embodiment, the functions of the controller 13 are realized by the processor executing calculation processing according to the control signal generation program 121 stored in the storage unit 12.

[0023] The target information acquisition unit 131 acquires images and videos (image information) captured by the camera 2 and audio (audio information) collected by the microphone 3 via the communication unit 11. The target information acquisition unit 131 acquires image information and audio information of the target person T1 (infant) as information relating to the person being soothed. The target information acquisition unit 131 stores the acquired image information and audio information in the storage unit 12 as necessary for subsequent processing. The target information acquisition unit 131 acquires this information at substantially the same time and for a suitable time length for subsequent processing (a suitable time length for detecting and estimating irritability and emotions), and stores it in the storage unit 12 as a single dataset. This data is used to detect and estimate the irritability and emotions of the target person T1 at that time (time period).

[0024] The target information acquisition unit 131 may also acquire the subject T1's biometric information (biosignals) in order to estimate the subject T1's moodiness and emotions. In this case, a sensing device capable of detecting the subject T1's biometric information (biosignals) must be installed on the subject T1. The subject T1's biometric information (biosignals) may include, for example, electroencephalogram (EEG) information (EEG signals) and heart rate information (heart rate signals). The EEG information and heart rate information are stored in the memory unit 12. The target information acquisition unit 131 acquires biometric information (EEG information and heart rate information) that is substantially the same time and of a suitable duration for subsequent processing (a suitable duration for detecting and estimating moodiness and emotions), and stores it in the memory unit 12 as a single dataset. This data is used to detect and estimate the subject T1's moodiness and emotions at that time (time period).

[0025] Furthermore, if the target information acquisition unit 131 also uses image information and audio information for subsequent processing, it acquires image information, audio information, and biometric information and stores them in the storage unit 12 as a single dataset.

[0026] Furthermore, the target information acquisition unit 131 acquires prior information about the target person T1 as information for determining the status of the target person T1. The target information acquisition unit 131 acquires information about the target person T1 that has been manually entered in advance by the target person T1's guardian (driver D1) or information about the target person T1 stored by an application on a terminal device such as a smartphone (via the communication unit 11), and stores it in the storage unit 12 (prior information table 122) as prior information about the target person T1.

[0027] Figure 3 shows an example of a pre-information table 122. As shown in Figure 3, the items in the pre-information table 122 include a "data ID," which is identification information for identifying the pre-information dataset. A data ID is set for each subject T1, and a data record is constructed for each data ID.

[0028] Furthermore, the pre-information table 122 includes information such as subject T1's "age," "sleep duration" and "wake-up time" related to recent sleep, recent "meal time," and recent "diaper change time." The pre-information table 122 also includes "preferred image type" data, which shows image type data such as images of subject T1's family, including their mother, or images of animals, characters, etc. that subject T1 likes. The pre-information table 122 also includes "preferred voice type" data, which shows voice type data such as voices of people, characters, etc. that subject T1 likes. In other words, the pre-information table 122 includes subject T1's profile data (personal characteristic data that contributes to bad moods) and behavioral history data (recent past behavioral data that contributes to bad moods).

[0029] The state detection unit 132 detects (estimates) the unhappy state of subject T1 based on the image information, voice information, and biometric information of subject T1 acquired by the target information acquisition unit 131. More specifically, the state detection unit 132 performs analysis processing on the image information, voice information, and biometric information of subject T1 to detect (estimate) whether subject T1 is unhappy, and also the detailed state of emotion. The unhappy state detected (estimated) by the state detection unit 132 is stored in the data table of the storage unit 12.

[0030] The method for detecting an unhappy state by the state detection unit 132 can be, for example, the following method.

[0031] An AI (Artificial Intelligence) model is used to estimate the emotion (type and intensity) of subject T1 based on a facial image of the subject, and the subject T1's state is determined according to that emotion. Emotion can be estimated by applying emotion indices (e.g., arousal level and (autonomic nervous system) activity level) calculated from biometric information (e.g., electroencephalogram information and heart rate information) to Russell's emotion circle model. Therefore, by training the AI ​​model using training data consisting of estimated emotions based on biometric signals from the subject and facial images for generating training data, an AI model that estimates emotions based on the facial image of subject T1 is generated. Whether or not the subject is in a bad mood can be determined from the estimated emotion using, for example, a data table in which information on emotion type (and intensity) and whether or not the subject is in a bad mood is stored in association.

[0032] When detecting (estimating) the emotions of subject T1 based on speech information including the pronunciation of subject T1's voice, the state detection unit 132 performs analysis processing based on subject T1's voice volume, frequency, speech speed, frequency of speech, speech recognition, etc. The state detection unit 132 detects the emotions of subject T1 based on, for example, the waveform pattern of the voice, speech recognition results, volume change patterns, or changes in the voice (tempo, pitch range, etc.). More specifically, it detects the emotions of subject T1 using a data table in which these features of subject T1's voice and subject T1's emotions are associated and stored. It is also possible to estimate subject T1's unhappy state (emotion) using a trained AI model trained on training data consisting of the above-mentioned characteristics of the voice produced by the subject for generating training data and the subject's emotions at the time of production. Whether or not subject T1 is unhappy can be determined from the estimated emotion using, for example, a data table in which information on the type (and intensity) of emotion and whether or not subject T1 is unhappy is associated and stored.

[0033] Furthermore, when detecting (estimating) the emotions of subject T1 based on both image and audio information, methods such as using a data table in which facial image features and vocalization features are associated with emotions and stored accordingly, or using a trained emotion detection AI model trained on training data consisting of the subject's facial image features, vocalization features, and the subject's emotions in that state, can be applied.

[0034] Next, we will explain an example of an emotion estimation method used when estimating the emotions of subject T1 based on subject T1's biometric information (electroencephalogram information and heart rate information), or when estimating the emotions of a subject used in generating training data.

[0035] The emotion estimation model uses the subject T1's biometric information (electroencephalogram information and heart rate information) to estimate emotions based on two emotion index values, which are indicators of the subject's mental and physical state related to emotions. One of the emotion index values ​​used in this embodiment is the central nervous system arousal level (hereinafter referred to as arousal level), and its index value can be calculated from the "beta wave / alpha wave of the electroencephalogram." The other emotion index value is the autonomic nervous system activity level (hereinafter referred to as activity level), and its index value can be calculated from the "standard deviation of the heart rate LF (Low Frequency) component (low-frequency component of the heart rate waveform signal)."

[0036] The emotion estimation model consists of a model for estimating emotions based on arousal and activity levels (calculation formulas and conversion data tables). For example, the emotion estimation model consists of a two-dimensional model that estimates emotions using arousal and activity levels as parameters. This two-dimensional model is created based on medical evidence (such as research papers) that demonstrates the relationship between arousal and activity levels and emotions.

[0037] Figure 4 shows a two-dimensional model (psychological plane) which is an example of an emotion estimation model. In the psychological plane shown in Figure 4, the vertical axis represents "arousal level (aroused-unaroused)" and the horizontal axis represents "autonomic nervous system activity level (sympathetic nervous system activity (strong emotion)-parasympathetic nervous system activity (weak emotion))." In this psychological plane, each of the four quadrants separated by the vertical and horizontal axes is assigned a corresponding emotion type. The distance from each axis indicates the intensity of the corresponding emotion.

[0038] Furthermore, emotions can be estimated from the coordinates obtained by plotting two types of emotional indicator values ​​(arousal and activity levels) obtained based on biosignals onto a psychological plane. Specifically, emotions and their intensity can be estimated based on which quadrant the plotted coordinates lie in on the psychological plane, their position within that quadrant, and their distance from the origin.

[0039] It should be noted that the two-dimensional model (psychological plane) included in the emotion estimation model is not limited to the psychological plane shown in Figure 4; other models such as Russell's cyclic emotion model may also be used.

[0040] The cause analysis unit 133 analyzes the cause of the unhappy state detected by the state detection unit 132. Specifically, it analyzes the cause using the cause data table 124 shown in Figure 5. Figure 5 is a diagram showing an example of the cause data table 124.

[0041] The Cause Data Table 124 is a data table composed of parameters such as "emotion," "sleep level," "meal level," and "diaper level," and a value (solution) corresponding to each parameter value, called "cause of bad mood." Each value in the Cause Data Table 124 will be set to an appropriate value by the developer or others based on experiments, etc. For example, if "emotion" is "sadness," "sleep level" is "1," "meal level" is "1," and "diaper level" is "1," the analysis result for "cause of bad mood" will be "want to play."

[0042] Sleep level is data indicating the intensity of sleepiness in subject T1. It is calculated based on a formula or data table appropriately set based on experiments, etc., using, for example, the most recent sleep duration (shorter indicates stronger sleepiness) and the elapsed time since the end of the most recent sleep (wake-up time) (longer indicates stronger sleepiness).

[0043] The meal level is data indicating the hunger level of the subject T1, and is calculated based on a formula or data table appropriately set based on experiments, etc., for example, on the elapsed time since the most recent meal (the longer the time, the higher the hunger level).

[0044] The diaper level is data indicating the degree of discomfort a subject T1 experiences with diapers. It is calculated based on a formula or data table appropriately established based on experiments, for example, the elapsed time since the most recent diaper change (the longer the time, the higher the discomfort).

[0045] The cause analysis unit 133 reads data on subject T1's sleep duration, wake-up time, meal times, and diaper change times from the pre-information table 122, and calculates sleep level, meal level, and diaper level based on this data. The cause analysis unit 133 compares the emotion detected by the state detection unit 132 with the calculated sleep level, meal level, and diaper level in the cause data table 124 to determine the cause of the unhappy state corresponding to subject T1.

[0046] The cause analysis unit 133 can also be implemented using a cause analysis AI model. Specifically, a cause analysis AI model can be generated by training a pre-training AI model using training data that takes emotions, sleep duration, wake-up time, meal times, and diaper change times (or emotions, sleep level, meal level, and diaper level) as input values ​​and the causes of the unhappy state as the correct values. This cause analysis AI model is an AI model that outputs the causes of unhappiness when emotions, sleep duration, wake-up time, meal times, and diaper change times, or emotions, sleep level, meal level, and diaper level are input.

[0047] The corresponding action acquisition unit 134 acquires soothing actions corresponding to the cause of the unhappy state analyzed by the cause analysis unit 133. Specifically, it determines the soothing action using the soothing action data table 125 shown in Figure 6. Figure 6 is a diagram showing an example of the soothing action data table 125.

[0048] The Suspicious Behavior Data Table 125 is a data table composed of parameters "Possible Person to Handle" and "Cause of Displeasure," and corresponding values ​​(solutions) for each parameter, namely "Countermeasure (Handling Person)" and "Countermeasure (Suspicious Behavior)." The values ​​in the Suspicious Behavior Data Table 125 will be set by the developers or others based on experiments or other appropriate criteria.

[0049] "Persons capable of responding" refers to individuals capable of responding to the subject T1. This is set (stored in the memory unit 12) by, for example, input by the driver D1 when boarding vehicle Ca1, or by recognizing passengers (adults) in vehicle Ca1 through image recognition. "Cause of displeasure" is the reason for the displeasure and is compared with the cause of the displeasure of subject T1 analyzed by the cause analysis unit 133. "Responder action" is the action that the person capable of responding should take in response to the cause of displeasure. "Calming actions" are actions taken towards subject T1 using in-vehicle devices, such as the display device 4 and speaker 5 (image display, sound output).

[0050] According to the soothing action data table 125, for example, if the person who can respond is "driver only" and the cause of the subject T1's fussiness is "want to play + soiled diaper", the response will be "parking + diaper change", and the soothing action will be "image P1 + sound S1 + vibration V1".

[0051] Furthermore, the response action acquisition unit 134 may acquire soothing actions using a soothing action acquisition AI model that takes the cause of bad mood and the person who can provide the support as input. The soothing action recommendation AI can be generated by training a pre-training AI using a large amount of training data in which the cause of bad mood and the person who can provide the support are input data, and appropriate soothing actions for the situation of the said cause of bad mood and the person who can provide the support are the correct answer data.

[0052] The detailed operation acquisition unit 135 acquires detailed content data of the soothing operation acquired by the corresponding operation acquisition unit 134. For example, the detailed operation acquisition unit 135 acquires data such as image display content (display character, etc.), audio output content (dialogue, etc.), and vibration output content (vibration pattern, etc.) of the soothing operation. In this embodiment, detailed content data of the soothing operation is acquired using a generative AI implemented by LLM (Large Language Models), etc. In this embodiment, the generative AI model used is a model from an external server (connected via the communication unit 11), but it may also be installed (configured) within the control signal generation device 10.

[0053] Specifically, the detailed action acquisition unit 135 creates prompts for acquiring detailed actions to be output to the generation AI. The detailed action acquisition unit 135 obtains a prompt template from the prompt template table 123 in the memory unit 12. The template consists of a standard instruction sentence for creating detailed actions and parameter items to be inserted into the standard sentence, for example, "(standard sentence) + (parameter 1: soothing action) + (parameter 2: data on the subject's unhappy state) + (parameter 3~: data related to the soothing subject (selected from the pre-information table 122))".

[0054] The detailed action acquisition unit 135 inserts parameters into this prompt template to create a prompt to be input to the generative AI, and then sends the created prompt to the generative AI. Specifically, the detailed action acquisition unit 135 inserts the subject's unhappy state data detected by the state detection unit 132 as parameter 2. The detailed action acquisition unit 135 also inserts the subject's age, preferred image type, and preferred voice type from the pre-information table 122 as parameters 3, 4, and 5.

[0055] Therefore, for example, the detailed operation acquisition unit 135 is Please create detailed operational information that meets the following conditions (standard text). Target action (standard phrase): Image P1 + Audio S1 + Vibration V1 (Parameter 1) Subject's unhappy state (standard phrase): I want to play (parameter 2) Target age (standard text): 1 year old (parameter 3) Preferred image (pre-set text): Bear character A (parameter 4) Preferred voice (pre-set phrase): Dog character B (parameter 5) We create prompts like this and send them to the generative AI.

[0056] In response to this prompt, the generating AI sends detailed operation information for the target operation (image P1 + sound S1 + vibration V1) as a reply, and the detailed operation acquisition unit 135 acquires this detailed operation information. For example, if the generating AI has the function of generating image (video) data, sound data, and vibration data itself, the generating AI generates image (video) data, sound data, and vibration data that conform to the conditions described in the prompt and sends them to the generation signal generation device 10. For example, the generating AI creates video data of bear character A, vocal data of dog character B, and vibration data suitable for a 1-year-old child's playtime, sends them to the generation signal generation device 10, and the detailed operation acquisition unit 135 acquires this detailed operation information.

[0057] It is also preferable to add parameters such as changes in the surrounding environment that affect the infant's condition, emotions, etc. (e.g., noise, vibration, temperature). In this case, various data related to the surrounding environment can be detected by sensors, etc., and updated and stored in the memory unit 12, and the latest stored data related to the surrounding environment can be added to the prompt as parameters.

[0058] Furthermore, the prompts output to the generation AI may be generated by a prompt generation AI. This prompt generation AI receives input from the unhappy state acquired by the corresponding action acquisition unit 134, the various data about the subject in the prior information table 122 as described above, and the soothing actions acquired by the corresponding action acquisition unit 134, and generates prompts to cause the generation AI to respond with detailed action information.

[0059] Such prompt-generating AI can be created by training a pre-training AI model using a large amount of training data, which takes various data such as the aforementioned bad mood state, soothing actions, and information about the subject as input values, and appropriate prompts (created by the designer / developer, etc.) for those inputs as ground truth data.

[0060] It is also preferable to add environmental data, such as changes in the surrounding environment that affect the state and emotions of infants and toddlers (e.g., noise, vibration, temperature), to the learning data (input data) during training. In this case, the environmental data will also be input to the prompt generation AI when creating prompts.

[0061] The control signal generation unit 136 generates control signals for each output device based on the detailed operation information acquired by the detailed operation acquisition unit 135. More specifically, the control signal generation unit 136 generates image (video) data for the display device 4, audio data for the speaker 5, and vibration data for the vibration device 6 based on the detailed operation information. If the detailed operation information is in text data (when the generation system AI only has a text output function), a data conversion table is provided in the storage unit 12 (created in advance by the design and development team, etc., and stored in the storage unit 12), and this data conversion table is used to create image (video) data, audio data, and vibration data corresponding to the text data (detailed operation information).

[0062] In this case, the text data of the detailed operation information is subjected to linguistic analysis (such as extracting the text data registered in the data conversion table) and analyzed into data suitable for matching with the data conversion table. For example, if the text data of the detailed operation information contains the word "mama," the control signal generation unit 136 outputs an image signal to the display device 4 of an image (video) of a mother that has been pre-stored in the conversion table of the storage unit 12 and associated with "mama."

[0063] The supply unit 137 provides (outputs) the control signals generated by the control signal generation unit 136 to the display device 4, speaker 5, and vibration device 6. The audio data generated by the generation system AI as detailed information about the soothing actions is output to the subject T1 via speaker 5. The image (video) data generated by the generation system AI as detailed information about the soothing actions is output to the subject T1 via display device 4. The vibration data generated by the generation system AI as detailed information about the soothing actions is output to the subject T1 via vibration device 6.

[0064] According to the above configuration, when generating control signals to the display device 4, speaker 5, and vibration device 6 (output device) that perform soothing actions, a prompt containing information that does not require user input is automatically created and input to the generation AI, and the control signals are generated based on the response of the generation AI. Furthermore, since the control signals for soothing actions are not selected and generated from pre-prepared patterns, a wide variety of soothing actions can be provided to infants (target person T1). Therefore, it becomes possible to effectively soothe infants and appropriately resolve their fussy state. In addition, even when soothing actions are needed when the infant's guardian (driver D1) is unable to attend to the infant, the amount of user input required is suppressed, so soothing actions can be performed on the infant with little difficulty.

[0065] <3. Example of operation of the control signal generator> Figure 7 is a flowchart showing the control signal generation process performed by the control signal generation device 10. The operation shown in this flowchart is realized by a computer program (control signal generation program 121) executed by the controller 13 (the computer that constitutes the controller 13).

[0066] The computer program that implements the control signal generation method according to this embodiment in a computer device is installed in a computer device such as the control signal generation device 10 to realize the various functions described above. Furthermore, such a computer program is provided to the computer device via a computer-readable non-volatile recording medium. For example, optical discs on which the computer program is recorded are distributed and sold, or computer programs stored on the hard disk of a server device are distributed and sold via a network environment. In addition, the computer program that implements the control signal generation method according to this embodiment in a computer device may consist of only one program, or it may consist of multiple programs.

[0067] The process shown in Figure 7 begins, for example, when vehicle Ca1 starts up and camera 2 and microphone 3 begin acquiring images, video, and audio of the person being soothed T1 (infant), and when driver D1 requests the start of "monitoring mode" to monitor (perform soothing actions) the person T1. The process shown in Figure 7 is then repeatedly executed for the period during which "monitoring mode" is required (for example, for a period set by driver D1, until driver D1 performs the operation to deactivate "monitoring mode", or until vehicle Ca1 stops, etc.).

[0068] In step S101, the controller 13 (target information acquisition unit 131) of the control signal generation device 10 acquires prior information of the subject T1, updates the data in the prior information table 122 with the latest information using the acquired prior information, and proceeds to step S102. More specifically, the controller 13 (target information acquisition unit 131) receives information such as the sleep duration and wake time related to the most recent sleep, the most recent meal time, and the most recent diaper change time, and updates and stores it in the prior information table 122 of the storage unit 12.

[0069] In step S102, the controller 13 (target information acquisition unit 131) acquires image and audio information of subject T1 from the camera 2 and microphone 3, and proceeds to step S103. The acquired image and audio information is stored in the data table of the storage unit 12. When detecting the state of subject T1 based on the subject T1's biological information (biometric signals), the controller 13 acquires biological information (biometric signals) from a sensing device installed on subject T1 and stores it in the data table of the storage unit 12.

[0070] In step S103, the controller 13 (state detection unit 132) uses the image and audio information of subject T1 acquired in step S102 to detect (estimate) the emotions and moodiness state (whether or not the subject is in a bad mood). If the subject is in a bad mood, the process proceeds to step S104; otherwise, the process ends.

[0071] In step S104, the controller 13 (cause analysis unit 133) compares the emotion estimated in step S103 with the prior information of subject T1 obtained in step S101 (sleep, eating, diaper information, etc.) to the cause data table 124 to estimate the cause of the irritability, and then proceeds to step S105.

[0072] In step S105, the controller 13 (corresponding action acquisition unit 134) uses the cause of the unhappy state estimated in step S104 to compare with the soothing action data table 125 to determine countermeasures (handler action, soothing action), and then proceeds to step S106.

[0073] In step S106, the controller 13 (providing unit 137) notifies the responder of the measures to be taken by the responder on board vehicle Ca1, which were determined in step S105, using the display device 4 or speaker 5, and then proceeds to step S107.

[0074] In step S107, the controller 13 (detailed action acquisition unit 135) creates a prompt, which is the content of a question (input data to the generating AI) to ask the generating AI in order to obtain detailed action information regarding the soothing action among the countermeasures decided in step S105. The controller 13 then sends the created prompt to the generating AI and proceeds to step S108. Specifically, as described above, the prompt is generated by inserting various data such as the subject's moodiness state, age, and preferred images and sounds, which have been acquired and detected, into the prompt template.

[0075] By sending a prompt to the generation AI, the generation AI will send a response back. In step S108, the controller 13 (detailed operation acquisition unit 135) acquires (receives) this response from the generation AI and proceeds to step S109.

[0076] In step S109, the controller 13 (control signal generation unit 136, supply unit 137) generates control signals for each output device, specifically image signals, audio signals, and vibration signals, based on the response (detailed operation information) obtained in step S108, outputs these generated control signals to each output device, and then proceeds to step S110.

[0077] In step S110, the controller 13 (target information acquisition unit 131) acquires image information and audio information of the subject T1 from the camera 2 and microphone 3, and then proceeds to step S111.

[0078] In step S111, the controller 13 (state detection unit 132) uses the image and audio information of subject T1 acquired in step S110 to detect (estimate) subject T1's unhappy state (whether or not the subject is unhappy). If there is no change in the content of the unhappiness (the subject is unhappy, and the content has not changed), the device proceeds to step S112. If there is a change in the content of the unhappiness (the subject is unhappy, and the content has changed), the device proceeds to step S113. If the unhappiness has been resolved, the device proceeds to step S115.

[0079] In step S112, the controller 13 (state detection unit 132) determines whether the duration of bad mood exceeds a predetermined threshold time. If the duration of bad mood exceeds the predetermined threshold time, the device moves to step S114 and continues the process of determining the cause of bad mood and addressing the bad mood. If the duration of bad mood does not exceed the threshold time, the device moves to step S110 and continues the current bad mood management.

[0080] In step S113, the controller 13 (detailed operation acquisition unit 135) stores (updates) the current countermeasures (the actions taken by the responder and the soothing actions) and their effects (the unhappy state of the target person T1 before and after the countermeasures) in the storage unit 12 as additional information for the prompt, and then proceeds to step S104. Therefore, this additional information will be added to subsequent prompts, and the next countermeasures will be configured to take into account the effects of the current countermeasures.

[0081] In step S114, the controller 13 (detailed operation acquisition unit 135) stores (updates) information in the storage unit 12 as additional prompt information indicating that the current countermeasures (response by the responder and the content of the soothing actions) are ineffective, and then proceeds to step S104. Therefore, subsequent prompts will include this additional information, and the next countermeasures will be configured to take into account the effectiveness of the current countermeasures.

[0082] In step S115, the controller 13 (detailed operation acquisition unit 135) stores (updates) information in the storage unit 12 as additional prompt information indicating that the bad mood has been resolved by the current countermeasures (the actions taken by the person in charge and the content of the soothing actions), and terminates processing. Therefore, prompts for subsequent bad moods will include this additional information, and the next countermeasures will be configured to take into account the effects of the current countermeasures.

[0083] For example, a prompt might be structured as follows: "(Standard phrase) + (Parameter 1: Data of the person handling the soothing behavior) + (Parameter 2: Data on the target's unhappy state) + (Parameter 3~: Data related to the person being soothed (selected from pre-information table 122)) + (Parameter: Data on the target's unhappy state) + (Parameter: Data on the soothing behavior) + (Additional information)".

[0084] Therefore, specifically, the detailed operation acquisition unit 135 is, "Please create detailed operational information that meets the following conditions (standard text). Note that the countermeasure A (previous detailed countermeasure) did not resolve the subject's unhappy state." Target action (standard phrase): Image P1 + Audio S1 + Vibration V1 (Parameter 1) Subject's unhappy state (standard phrase): I want to play (parameter 2) Target age (standard text): 1 year old (parameter 3) Preferred image (pre-set text): Bear character A (parameter 4) Preferred voice (pre-set phrase): Dog character B (parameter 5) We create prompts like this and send them to the generative AI.

[0085] Furthermore, by organizing and storing the additional information stored in steps S113, S114, and S115, such as by stratifying it according to the state of unhappiness, it becomes possible to use it as additional information for prompts for various states of unhappiness thereafter, thereby improving the accuracy of decisions regarding countermeasures for states of unhappiness. Moreover, by organizing and storing this additional information according to the individual, it becomes possible to select and use it as additional information for prompts appropriate to the individual, further improving the accuracy of decisions regarding countermeasures for states of unhappiness.

[0086] <4. Things to keep in mind> The various technical features disclosed as embodiments herein can be modified in various ways without departing from the spirit of the technical creation. That is, the above embodiments are illustrative in all respects and not restrictive. The technical scope of the present invention is indicated by the claims rather than by the above descriptions of embodiments, and includes all modifications that fall within the meaning and scope equivalent to the claims. Furthermore, the multiple embodiments shown herein may be combined as appropriate to the extent possible.

[0087] Furthermore, although the above embodiment explains that various functions are implemented in software through CPU arithmetic processing according to a program, at least some of these functions may be implemented by electrical hardware resources. These hardware resources may be implemented entirely or partially by, for example, ASICs (Application Specific Integrated Circuits) or FPGAs (Field Programmable Gate Arrays). Conversely, at least some of the functions implemented by hardware resources may be implemented in software.

[0088] Furthermore, the system may include a computer program that enables a processor (computer) to implement at least some of the functions of the Ayashi provision system 1 (control signal generation device 10). Such a computer program can be stored on a computer-readable non-volatile recording medium (for example, in addition to the non-volatile memory mentioned above, it can also be provided (sold, etc.) on an optical recording medium (for example, an optical disc), a magneto-optical recording medium (for example, a magneto-optical disc), a USB memory, or an SD card, etc.), and it can also be provided from a server device via a communication line such as the Internet, a method known as download provision. [Explanation of symbols]

[0089] 1. Ayashi Provision System 2 cameras 3 Microphones 4 Display device (output device) 5. Speaker (Output device) 6. Vibration device (output device) 10 Control signal generator 11 Communications Department 12 Storage section 13 Controllers 20 Generation AI device 121 Control signal generation program 122 Pre-information Table 123 Prompt Template Table 124 Cause of Occurrence Data Table 125 Suspicious Action Data Table 131 Target Information Acquisition Unit 132 State detection unit 133 Cause Analysis Department 134 Corresponding operation acquisition unit 135 Detailed operation acquisition section 136 Control signal generation unit 137 Provision Department D1 Driver T1 Target Group Ca1 Vehicle

Claims

1. A control signal generating device that generates control signals to be output to an output device that performs soothing actions on a target person, comprising a controller, The aforementioned controller, The subject's unhappy state is detected, A prompt is created to cause the generation AI to generate information on soothing actions corresponding to the detected unhappy state. The generated prompt is output to the generation AI, The AI ​​generation system obtains the response to the prompt, Based on the obtained response, the control signal is generated. Control signal generator.

2. The aforementioned controller, The subject's state of displeasure is estimated based on emotions estimated from the subject's image or biosignals. The control signal generation device according to claim 1.

3. The aforementioned controller, The prompt includes subject information based on the subject's profile data and behavioral history data. A control signal generation device according to claim 1 or claim 2.

4. The aforementioned controller, The subject's state of displeasure is estimated based on the subject's image or the subject's biosignals, Based on the estimated emotion, the subject's profile data, and behavioral history data, the cause of the subject's bad mood is estimated. The estimated cause is included in the prompt. The control signal generation device according to claim 1.

5. The aforementioned controller, We obtain information on the responders who are capable of dealing with the aforementioned target person, Based on the aforementioned cause, the response measures and actions to be taken by the responder in the countermeasure against soothing the person concerned will be determined. The details of the actions taken by the responding person are communicated using the output device. The prompt is created by including the soothing action content in the prompt, causing the generation system AI to generate the detailed content of the soothing action content. The control signal generation device according to claim 4.

6. A method for generating a control signal to output to an output device that performs soothing actions on a target person, The subject's unhappy state is detected, A prompt is created to cause the generation AI to generate information on soothing actions corresponding to the detected unhappy state. The generated prompt is output to the generation AI, The AI ​​generation system obtains the response to the prompt, Based on the obtained response, the control signal is generated. Method for generating control signals.

7. A control signal generation program that generates control signals to be output to an output device that performs soothing actions on a target person, The subject's unhappy state is detected, A prompt is created to cause the generation AI to generate information on soothing actions corresponding to the detected unhappy state. The generated prompt is output to the generation AI, The AI ​​generation system obtains the response to the prompt, The process of generating the control signal based on the acquired response is as follows: Let the computer do it. Control signal generation program.

8. A soothing service system that performs soothing actions on a target person, The system comprises a control signal generation device and an output device that performs the soothing action on the target person, The control signal generating device is The subject's unhappy state is detected, A prompt is created to cause the generation AI to generate information on soothing actions corresponding to the detected unhappy state. The generated prompt is output to the generation AI, The AI ​​generation system obtains the response to the prompt, Based on the acquired response, the control signal is generated. The generated control signal is output to the output device. The output device is The control signal output by the control signal generation device is acquired, Perform the soothing action on the target person in accordance with the acquired control signal. A system for providing auspicious information.

Images