Control system, control device, learning model generation method, control method, and program

Abstract

To provide a comfortable environment corresponding to an individual.SOLUTION: A control system comprises: a sensor 300 which detects a user's vital sign; and a control apparatus 100 which acquires individual identification information for identifying an individual user, generates a learning model by machine learning about a control content of an electric apparatus 200 corresponding to the vital sign for each piece of the acquired individual identification information, inputs the acquired individual identification information and the vital sign detected by the sensor 300 to the learning model and thereby acquires control information indicating the control content from the learning model, and controls the electric apparatus 200 by using the control content indicated by the acquired control information.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a control system, a control device, a learning model generation method, a control method, and a program. [Background technology]

[0002] In recent years, technologies have been devised that detect biometric information (vital signs) such as age, sex, body temperature, heart rate, and respiratory rate of a user of environmental control equipment such as lighting fixtures and air conditioners, and control the environmental control equipment based on the detected vital signs (see, for example, Patent Document 1). In such technologies, the detected vital signs are input into a trained model that has undergone machine learning to determine the control content of the environmental control equipment in response to the vital signs, thereby acquiring the control content, and the environmental control equipment is controlled using the acquired control content. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2021-093267 Summary of the Invention [Problem to be solved by the invention]

[0004] The relationship between vital signs and an individual's evaluation of the results of control using those vital signs varies from person to person. In other words, when the same vital signs are detected from multiple users and environmental control equipment is controlled based on the detected vital signs, as in the above-mentioned technology, some users may find the control comfortable, while others may find it uncomfortable. As such, control based solely on vital signs that are not linked to individuals cannot necessarily provide an optimal environment. In other words, there is a risk that the environment provided may be uncomfortable for some users.

[0005] An object of the present invention is to provide a control system, a control device, a learning model generation method, a control method, and a program that can provide a comfortable environment tailored to each individual. [Means for solving the problem]

[0006] The control system of the present invention comprises: a personal information acquisition unit that acquires personal identification information that identifies an individual user; a sensor for detecting a vital sign of the user; a control unit that controls the electrical equipment; a learning model generation unit that performs machine learning on the control content of the electrical device corresponding to the vital sign for each of the individual identifying information acquired by the personal information acquisition unit to generate a learning model; an inference unit that inputs the individual identifying information acquired by the individual information acquisition unit and the vital signs detected by the sensor into the learning model, and thereby acquires control information indicating the control content from the learning model; The control unit controls the electrical appliance using the control content indicated by the control information acquired by the inference unit.

[0007] The control device of the present invention also includes: a personal information acquisition unit that acquires personal identification information that identifies an individual user; a control unit that controls the electrical equipment; a learning model generation unit that performs machine learning on the control content of the electrical device corresponding to the vital signs of the individual user detected by a sensor for each of the individual identifying information acquired by the personal information acquisition unit to generate a learning model; an inference unit that inputs the individual identifying information acquired by the individual information acquisition unit and the vital signs detected by the sensor into the learning model, and thereby acquires control information indicating the control content from the learning model; The control unit controls the electrical appliance using the control content indicated by the control information acquired by the inference unit.

[0008] Further, the learning model generation method of the present invention includes: A process of acquiring personal identification information that identifies an individual user and the vital signs of the individual user; generating training data that associates the acquired personal identification information of the individual user, the vital signs of the individual user, and control information indicating control details of the electrical appliances; Machine learning is performed using the generated training data to generate a learning model that outputs the control information according to the personal identification information and the vital signs of the individual user.

[0009] Further, the control method of the present invention includes: A process of acquiring personal identification information that identifies an individual user and the vital signs of the individual user; a process of inputting the acquired individual identifying information and the vital signs into a learning model that has undergone machine learning to determine control content of electrical appliances suitable for the vital signs of each individual user for each of the acquired individual identifying information of the individual user, and thereby acquiring control information indicating the control content from the learning model; The control device performs a process of controlling the electrical device using the control content indicated by the acquired control information.

[0010] The program of the present invention also includes: A program to be executed by a computer, A procedure for acquiring personal identification information that identifies an individual user and the vital signs of the individual user; generating training data that associates the acquired personal identification information of the individual user, the individual user's vital signs, and control information indicating control details of the electrical appliances; The generated training data is used to generate a learning model that outputs the control information according to the individual identification information and the vital signs of the individual user.

[0011] The program of the present invention also includes: A program to be executed by a computer, A procedure for acquiring personal identification information that identifies an individual user and a vital sign of the individual user; a step of inputting the acquired individual identifying information and the vital signs into a learning model that has undergone machine learning to determine control content of an electrical device suitable for the vital signs of each individual user for each of the acquired individual identifying information of the individual user, thereby acquiring control information indicating the control content from the learning model; The control device executes a procedure for controlling the electrical device using the control content indicated by the acquired control information. [Effects of the Invention]

[0012] In the present invention, a comfortable environment tailored to the individual can be provided. [Brief explanation of the drawings]

[0013] [Figure 1] FIG. 1 is a diagram illustrating a first embodiment of a control system according to the present invention. [Figure 2] 2 is a diagram illustrating an example of an internal configuration of a control device illustrated in FIG. 1. FIG. [Figure 3] 2 is a flowchart illustrating an example of a learning model generation method in the learning phase of the control system shown in FIG. [Figure 4] 2 is a sequence diagram for explaining an example of a control method in an inference phase of the control system shown in FIG. 1. FIG. [Figure 5] FIG. 2 is a sequence diagram illustrating an example of a reinforcement learning method in the control system shown in FIG. [Figure 6] FIG. 2 is a diagram illustrating a second embodiment of a control system according to the present invention. [Figure 7] 7 is a diagram illustrating an example of the internal configuration of the control device illustrated in FIG. 6. FIG. [Figure 8] 7 is a sequence diagram for explaining an example of a control method in an inference phase of the control system shown in FIG. 6. FIG. [Figure 9] FIG. 10 is a diagram illustrating a third embodiment of the control system of the present invention. [Figure 10] 10 is a diagram illustrating an example of the internal configuration of the control device illustrated in FIG. 9. FIG. [Figure 11]10 is a sequence diagram for explaining an example of a control method in an inference phase of the control system shown in FIG. 9. FIG. [Figure 12] FIG. 10 is a diagram illustrating a fourth embodiment of the control system of the present invention. [Figure 13] 13 is a diagram illustrating an example of the internal configuration of the control device illustrated in FIG. 12. FIG. [Figure 14] 13 is a sequence diagram for explaining an example of a control method in an inference phase of the control system shown in FIG. 12. FIG. DETAILED DESCRIPTION OF THE INVENTION

[0014] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. (First embodiment)

[0015] Fig. 1 is a diagram showing a first embodiment of a control system of the present invention. As shown in Fig. 1, the control system in this embodiment includes a control device 100, an electric device 200, a sensor 300, an imaging device 400, and a communication device 500. These components are connected to each other so as to be able to communicate with each other via a communication network 600. Alternatively, these components may be connected to each other directly without using the communication network 600.

[0016] The electrical device 200 is placed in a predetermined space and controls the environment in that space. The electrical device 200 is, for example, an air conditioner. The electrical device 200 is controlled according to the control content indicated by a control signal transmitted from the control device 100. When the electrical device 200 is an air conditioner, the temperature, humidity, etc. set in the electrical device 200 are controlled. The electrical device 200 may be of a type that is placed on a floor or a stand, or may be of a type that is installed on a wall or a ceiling.

[0017] The sensor 300 detects an individual's vital signs. The vital signs detected by the sensor 300 are biological information of the individual, such as body temperature, respiration, respiratory rate, pulse, pulse rate, heartbeat, pulse variability, pulse interval, pulse cycle, heart rate, heartbeat variability, heartbeat interval, heartbeat cycle, electrocardiogram, electroencephalogram, blood pressure, facial expression, skin potential, eye movement, blood oxygen concentration, epidermal temperature, and voice. The sensor 300 may be provided in a wearable device, such as a wristwatch, that is attached to the individual in advance. The sensor 300 may also detect vital signs by contacting or approaching the individual. The sensor 300 may also capture an image of the individual and detect vital signs based on the captured image. The sensor 300 may also detect vital signs from the individual using microwaves or the like without contacting the individual. The sensor 300 transmits the detected vital signs to the control device 100.

[0018] Imaging device 400 is a camera that captures images of each individual user. Imaging device 400 may be a camera that captures still images, or may be a camera that captures images at intervals equal to or shorter than a preset time interval (for example, a video camera that continuously captures images). Imaging device 400 transmits image data representing the captured images to control device 100. Imaging device 400 may be installed in a position where it can capture images of individuals entering an area where electrical device 200 operates.

[0019] The communication device 500 is owned by a user and inputs information based on an operation received from outside. The communication device 500 transmits the input information to the control device 100. The communication device 500 may be, for example, a mobile terminal such as a smartphone. Furthermore, if the communication device 500 pre-stores personal identification information that identifies the user who owns the device, the communication device 500 transmits the stored personal identification information to the control device 100 at a predetermined timing.

[0020] The control device 100 controls the electrical device 200. FIG. 2 is a diagram showing an example of the internal configuration of the control device 100 shown in FIG. 1. As shown in FIG. 2, the control device 100 shown in FIG. 1 has a personal information acquisition unit 110, a learning model generation unit 120, a learning model 130, an inference unit 140, a control unit 150, and a subjective information acquisition unit 160. Note that FIG. 2 shows only the main components related to this embodiment among the components included in the control device 100 shown in FIG. 1. Furthermore, the learning model generation unit 120 and the learning model 130 may be provided outside the control device 100.

[0021] The personal information acquisition unit 110 acquires personal identification information that identifies an individual user. The personal identification information is information uniquely assigned to the user and may be information that can identify the user. The personal information acquisition unit 110 may acquire personal identification information transmitted from the communication device 500. The personal information acquisition unit 110 may acquire personal identification information by performing face recognition processing on the user's face included in the image represented by the image data transmitted from the imaging device 400. The face recognition processing performed at this time may be processing using general image analysis and registered images, and is not particularly limited. In the learning phase, the personal information acquisition unit 110 may acquire personal identification information that is directly input to the control device 100.

[0022] During the learning phase of the learning model 130, the learning model generation unit 120 generates training data that associates the individual identifying information acquired by the personal information acquisition unit 110, vital signs input from an external source or transmitted from the sensor 300, and control information indicating the control content of the electrical appliance 200. For example, the learning model generation unit 120 generates training data that associates the individual vital signs of each user with control information indicating control content such as "lower the set temperature," "increase the set temperature," or "start dehumidification operation" based on subjective feelings such as "hot," "cold," or "humid." The same applies to the amount of temperature decrease (increase) and the operating time of the electrical appliance 200. The learning model generation unit 120 uses the generated training data to perform machine learning to generate the learning model 130 that outputs control information corresponding to the individual identifying information and the individual's vital signs. Because vital signs vary from person to person, the learning model generation unit 120 generates a learning model by performing machine learning on control content corresponding to the vital signs for each individual identifying information. For example, when heart rate is used as a vital sign, normal heart rates vary from user to user. Furthermore, when a heart rate detected as a vital sign is higher than normal, whether the heart rate is perceived as discomfort or comfort varies from user to user. The amount of change (increase or decrease) also varies from user to user. The same applies to other vital signs (body temperature, respiratory rate, pulse rate, pulse variability, pulse interval, pulse cycle, heart rate variability, heart rate interval, heart cycle, electrocardiogram, electroencephalogram, blood pressure, facial expression, skin potential, eye movement, blood oxygen concentration, epidermal temperature, and voice). Therefore, in the present invention, control content is not uniformly determined based on the detected vital sign, but is acquired from the learning model 130 for each individual. Furthermore, the learning model generation unit 120 uses the subjective information acquired by the subjective information acquisition unit 160 to perform machine learning on control content corresponding to the vital sign for each individual identifying information acquired by the personal information acquisition unit 110, thereby updating the learning model 130 (reinforcement learning). The subjective information will be described later.

[0023] The learning model 130 is a trained model that has learned the relationship between personal identification information, vital signs, and control contents. When the personal identification information and vital signs are input, the learning model 130 outputs control contents. The learning model 130 may have, for example, a neural network structure in which multiple neurons are interconnected. A neuron is an element that performs a predetermined calculation on multiple inputs and outputs one value as the calculation result. The learning model 130 is stored in a memory unit (not shown).

[0024] The inference unit 140 inputs the personal identification information acquired by the personal information acquisition unit 110 and the vital signs transmitted from the sensor 300 into the learning model 130, thereby acquiring control information indicating the control content of the electrical device 200 from the learning model 130. The inference unit 140 outputs the acquired control information to the control unit 150.

[0025] The control unit 150 controls the electric device 200 using the control content indicated by the control information output from the inference unit 140. Specifically, the control unit 150 transmits a control signal to the electric device 200 to be controlled, for controlling the operation of the electric device 200.

[0026] The subjective information acquiring unit 160 acquires subjective information from an external source. The subjective information is information indicating an evaluation input by a user who is using or has used a controlled environment using the electric device 200. The subjective information may be input by the user operating the communication device 500 and being received by the communication device 500 based on the operation. The communication device 500 transmits the input subjective information to the control device 100, and the subjective information acquiring unit 160 acquires the subjective information. Alternatively, the subjective information may be transmitted to the control device 100 from a remote control or a predetermined switch attached to the electric device 200 based on an operation on the remote control or the switch. For example, the user may operate a smartphone, which is the communication device 500, to input an evaluation of the environment used as subjective information, or the user may operate a remote control to input a subjective feeling, such as whether the environment used is hot or cold.

[0027] The following describes a method for generating a learning model in the learning phase of the control system shown in Fig. 1. Fig. 3 is a flowchart for explaining an example of a method for generating a learning model in the learning phase of the control system shown in Fig. 1.

[0028] First, the learning model generation unit 120 acquires vital signs (step S1). At this time, the vital signs may be detected using the sensor 300 and transmitted, or may be directly input to the control device 100. The learning model generation unit 120 also acquires personal identification information (step S2). At this time, the personal identification information may be input to the communication device 500 and transmitted, or may be directly input to the control device 100. The learning model generation unit 120 also acquires control information indicating the control content of the electrical appliance (step S3). At this time, the control information may be input to the communication device 500 together with the personal identification information input in step S2 and transmitted, or may be directly input to the control device 100. The order of the processes of steps S1 to S3 is not limited. Next, the learning model generation unit 120 generates training data that associates the acquired personal identification information of the individual, the personal vital signs, and the control information indicating the control content of the electrical appliance (step S4).

[0029] In actuality, the processing of steps S1 to S4 involves the learning model generation unit 120 detecting vital signs using the sensor 300 for a user with personal identification information acquired by the control device 100, adjusting the control information to an environment (temperature, humidity, etc.) that the individual feels comfortable in, and generating training data that associates the personal identification information, vital signs, and control content (e.g., lowering or raising the set temperature) at that time. The control information may be adjusted by the user with the personal identification information using the communication device 500 or the above-mentioned remote control or switch. The learning model generation unit 120 then performs machine learning using the generated training data to generate a learning model that outputs control information corresponding to the personal identification information and the individual's vital signs (step S5). This generates a learning model for acquiring control content corresponding to the vital signs of each individual user. Needless to say, more repeated processing of steps S1 to S5 can result in more accurate training data.

[0030] The following describes a control method in the inference phase of the control system shown in Fig. 1. Fig. 4 is a sequence diagram for explaining an example of a control method in the inference phase of the control system shown in Fig. 1.

[0031] First, the imaging device 400 captures an image of the face of a user entering an area where the electrical device 200 operates (step S11). Image data representing the captured image is then transmitted from the imaging device 400 to the control device 100 (step S12). The timing at which the imaging device 400 captures the image of the user's face is not particularly limited. This timing may be, for example, when a predetermined sensor detects that the user has entered the area where the electrical device 200 operates, or when the user performs a predetermined operation on the imaging device 400. The personal information acquisition unit 110 then performs a facial recognition process on the image represented by the image data transmitted from the imaging device 400 to identify the individual user (step S13). The facial recognition process performed here may be a commonly used process, such as extracting facial features contained in the image and comparing them with pre-registered facial features to identify the individual user.

[0032] Furthermore, the sensor 300 detects the vital signs of the user whose face image has been captured by the imaging device 400 (step S14). The vital signs detected by the sensor 300 are as described above. The sensor 300 transmits information indicating the detected vital signs to the control device 100 (step S15).

[0033] Next, when the inference unit 140 inputs the individual identifying information identified by the individual information acquiring unit 110 and the vital signs indicated by the information transmitted from the sensor 300 to the learning model 130 (step S16), the inference unit 140 acquires the control content of the electric device 200 from the learning model 130 (step S17). Then, the control unit 150 controls the electric device 200 in accordance with the control content acquired by the inference unit 140 from the learning model 130 (step S18).

[0034] The following describes a reinforcement learning method in the control system shown in Fig. 1. Fig. 5 is a sequence diagram for explaining an example of the reinforcement learning method in the control system shown in Fig. 1. This processing may be processing performed after a user uses an area in which electrical device 200 is controlled using control content inferred using learning model 130 as described above, or may be processing performed while the user is using that area.

[0035] When the communication device 500 receives an external operation and inputs subjective information (step S21), the input subjective information and personal identification information are transmitted from the communication device 500 to the control device 100 (step S22). The personal identification information transmitted here may be newly input to the communication device 500, or may be identification information previously set uniquely for the communication device 500. The subjective information is as described above. The communication device 500 may display an input screen (for example, a question or a questionnaire) for inputting the subjective information, and prompt the user to input the subjective information.

[0036] When the individual identifying information and the subjective information are transmitted from the communication device 500, the subjective information acquisition unit 160 acquires the transmitted individual identifying information and subjective information, and the learning model generation unit 120 performs machine learning using the acquired individual identifying information and subjective information to update (reinforcement learning) the learning model 130 (step S23). At this time, the learning model generation unit 120 analyzes the content (e.g., sentences) indicated by the subjective information and reflects the analysis results in the learning model 130.

[0037] In this manner, in this embodiment, training data is generated that associates personal identification information, vital signs of each individual user, and control information indicating the control content of the electrical appliance. The generated training data is then used to generate a learning model that outputs control information corresponding to the personal identification information and vital signs of each individual user. Furthermore, the personal identification information and vital signs of each individual user identified by performing facial recognition processing on an image of the user captured using a camera are input into the learning model, whereby control information is acquired from the learning model, and the electrical appliance is controlled using the acquired control information. This allows for the provision of an environment in which the user feels comfortable. Furthermore, after or while the user uses the provided environment, the learning model is updated (reinforcement learning) using subjective information from the user. This allows the learning model to output control information that is more suited to individual preferences. (Second embodiment)

[0038] Fig. 6 is a diagram showing a second embodiment of a control system of the present invention. As shown in Fig. 6, the control system in this embodiment includes a control device 101, an electric device 200, a sensor 300, a reader 401, and a communication device 500. These components are connected to each other so as to be able to communicate with each other via a communication network 600. Alternatively, these components may be connected to each other directly without using the communication network 600. The electric device 200, the sensor 300, and the communication device 500 are the same as those in the first embodiment. The control system in this embodiment also includes a tag 411.

[0039] The tag 411 is attached to each individual user and is provided with the user's personal identification information. The tag 411 may have the personal identification information indicated in predetermined code information such as a barcode or a two-dimensional code, or may be an electronic tag in which the personal identification information is electronically stored. The tag 411 may be provided on a communication device or an ID card carried by the user.

[0040] Reader 401 reads the personal identification information attached to tag 411 from tag 411. If the personal identification information is represented by code information in tag 411, reader 401 is a barcode reader or a two-dimensional code reader. If the personal identification information is written electronically in tag 411, for example, if tag 411 is an RFID (Radio Frequency Identifier) ​​tag, reader 401 is an RFID reader that electronically reads the personal identification information from tag 411. When a user holds tag 411 over reader 401, reader 401 reads the personal identification information from tag 411. Reader 401 transmits the read personal identification information to control device 101. Reader 401 may be installed in a position (for example, an entrance or the like) where it can read the personal identification information from tag 411 attached to a user who enters an area where electrical device 200 operates.

[0041] The control device 101 controls the electrical device 200. FIG. 7 is a diagram showing an example of the internal configuration of the control device 101 shown in FIG. 6. As shown in FIG. 7, the control device 101 shown in FIG. 6 has a personal information acquisition unit 111, a learning model generation unit 120, a learning model 130, an inference unit 140, a control unit 150, and a subjective information acquisition unit 160. Note that FIG. 7 shows only the main components related to this embodiment among the components included in the control device 101 shown in FIG. 6. The learning model generation unit 120, the learning model 130, the inference unit 140, the control unit 150, and the subjective information acquisition unit 160 are the same as those in the first embodiment.

[0042] The personal information acquisition unit 111 acquires personal identification information that identifies an individual user. The personal identification information is the same as that in the first embodiment. In the learning phase, the personal information acquisition unit 111 may acquire personal identification information transmitted from the communication device 500, may acquire personal identification information directly input to the control device 101, or may acquire personal identification information transmitted from the reader 401. In the inference phase, the personal information acquisition unit 111 acquires personal identification information transmitted from the reader 401.

[0043] A control method in the inference phase of the control system shown in Fig. 6 will be described below. Fig. 8 is a sequence diagram for explaining an example of a control method in the inference phase of the control system shown in Fig. 6. Note that the learning model generation method in the learning phase of the control system shown in Fig. 6 may be the same as the method in the first embodiment.

[0044] First, reader 401 reads personal identification information from tag 411 attached to a user entering an area where electrical device 200 operates (step S31), and the read personal identification information is transmitted from reader 401 to control device 101 (step S32). The timing at which reader 401 reads personal identification information from tag 411 is not particularly limited.

[0045] Furthermore, the sensor 300 detects the vital signs of the user to whom the tag 411, from which the reader 401 has read the personal identification information, is attached (step S33). The vital signs detected by the sensor 300 are as described above. The sensor 300 transmits information indicating the detected vital signs to the control device 101 (step S34).

[0046] Next, when the inference unit 140 inputs the individual identifying information transmitted from the reader 401 and acquired by the individual information acquisition unit 111 and the vital signs indicated by the information transmitted from the sensor 300 to the learning model 130 (step S35), the inference unit 140 acquires the control content of the electric device 200 from the learning model 130 (step S36). Then, the control unit 150 controls the electric device 200 in accordance with the control content acquired by the inference unit 140 from the learning model 130 (step S37).

[0047] The reinforcement learning method in the control system shown in FIG. 6 may be the same as the method in the first embodiment.

[0048] In this manner, in this embodiment, training data is generated that associates personal identification information, vital signs of each individual user, and control information indicating the control content of the electrical appliance. The generated training data is used to generate a learning model that outputs control information corresponding to the personal identification information and vital signs of each individual user. Furthermore, by inputting the personal identification information and vital signs of each individual user read from the tag using a reader into the learning model, control information is acquired from the learning model, and the electrical appliance is controlled using the acquired control information. This makes it possible to provide an environment in which the user feels comfortable. Furthermore, after or while the user uses the provided environment, the learning model is updated (reinforcement learning) using subjective information from the user. This allows the learning model to output control information that is more suited to individual preferences. (Third embodiment)

[0049] Fig. 9 is a diagram showing a third embodiment of the control system of the present invention. As shown in Fig. 9, the control system in this embodiment includes a control device 102, an electric device 200, a sensor 300, a microphone 402, and a communication device 500. These components are connected to each other so that they can communicate with each other via a communication network 600. Alternatively, these components may be connected to each other directly without using the communication network 600. The electric device 200, the sensor 300, and the communication device 500 are the same as those in the first embodiment.

[0050] Microphone 402 is a sound collector that collects surrounding sounds and voices (hereinafter referred to as "voice"). Microphone 402 transmits voice data indicating the collected voice to control device 102. Microphone 402 may be installed in a position where it can collect voice from users entering an area where electrical device 200 operates. The voice collected by microphone 402 is not limited to the voice of an individual user, but may also be a sound that can identify an individual user. The voice collected by microphone 402 may be, for example, the sound of a user's footsteps when walking. Furthermore, microphone 402 may be built into a device carried by the individual user, for example, communication device 500.

[0051] The control device 102 controls the electrical device 200. FIG. 10 is a diagram showing an example of the internal configuration of the control device 102 shown in FIG. 9. As shown in FIG. 10, the control device 102 shown in FIG. 9 has a personal information acquisition unit 112, a learning model generation unit 120, a learning model 130, an inference unit 140, a control unit 150, and a subjective information acquisition unit 160. Note that FIG. 10 shows only the main components related to this embodiment among the components included in the control device 102 shown in FIG. 9. The learning model generation unit 120, the learning model 130, the inference unit 140, the control unit 150, and the subjective information acquisition unit 160 are the same as those in the first embodiment.

[0052] The personal information acquisition unit 112 acquires personal identification information that identifies an individual user. The personal identification information is the same as that in the first embodiment. In the learning phase, the personal information acquisition unit 112 may acquire personal identification information transmitted from the communication device 500, or may acquire personal identification information directly input to the control device 102. In the learning phase, the personal information acquisition unit 112 may acquire personal identification information by performing speech recognition processing on the voice represented by the voice data transmitted from the microphone 402. The speech recognition processing performed at this time may be processing using general speech analysis and registered voice, and is not particularly limited. In the inference phase, the personal information acquisition unit 112 acquires personal identification information by performing speech recognition processing on the voice represented by the voice data transmitted from the microphone 402.

[0053] A control method in the inference phase of the control system shown in Fig. 9 will be described below. Fig. 11 is a sequence diagram for explaining an example of a control method in the inference phase of the control system shown in Fig. 9. Note that the learning model generation method in the learning phase of the control system shown in Fig. 9 may be the same as the method in the first embodiment.

[0054] First, microphone 402 collects the voice of a user entering the area where electrical device 200 operates (step S41), and voice data representing the voice is transmitted from microphone 402 to control device 102 (step S42). The timing at which microphone 402 collects the user's voice is not particularly limited. This timing may be, for example, when a predetermined sensor detects that the user has entered the area where electrical device 200 operates, or when the user performs a predetermined operation on microphone 402 (for example, speaking into microphone 402). Then, personal information acquisition unit 112 performs voice recognition processing on the voice represented by the voice data transmitted from microphone 402 to identify the individual user (step S43). The voice recognition processing performed here may be a commonly used process, such as extracting voice features and comparing them with pre-registered voice features to identify the individual.

[0055] Furthermore, the sensor 300 detects the vital signs of the user who uttered the voice indicated by the voice data used for identification by the personal information acquisition unit 112 (step S44). The vital signs detected by the sensor 300 are as described above. The sensor 300 transmits information indicating the detected vital signs to the control device 102 (step S45).

[0056] Next, when the inference unit 140 inputs the individual identifying information identified by the individual information acquiring unit 112 and the vital signs indicated by the information transmitted from the sensor 300 to the learning model 130 (step S46), the inference unit 140 acquires the control content of the electric device 200 from the learning model 130 (step S47). Then, the control unit 150 controls the electric device 200 in accordance with the control content acquired by the inference unit 140 from the learning model 130 (step S48).

[0057] The reinforcement learning method in the control system shown in FIG. 9 may be the same as the method in the first embodiment.

[0058] In this manner, in this embodiment, training data is generated that associates personal identification information, the user's individual vital signs, and control information indicating the control content of the electrical appliance. The generated training data is then used to generate a learning model that outputs control information corresponding to the personal identification information and the user's individual vital signs. Furthermore, the personal identification information and the user's individual vital signs identified by performing voice recognition processing on the user's voice collected using a microphone are input into the learning model, whereby control information is acquired from the learning model, and the electrical appliance is controlled using the acquired control information. This makes it possible to provide an environment in which the user feels comfortable. Furthermore, after or while the user uses the provided environment, the learning model is updated (reinforcement learning) using subjective information from the user. This allows the learning model to output control information that is more suited to individual preferences. (Fourth embodiment)

[0059] Fig. 12 is a diagram showing a fourth embodiment of the control system of the present invention. As shown in Fig. 12, the control system in this embodiment includes a control device 103, an electric device 200, sensors 301-1 to 301-n (n is a natural number), and a communication device 500. These components are connected to each other so that they can communicate with each other via a communication network 600. Alternatively, these components may be connected to each other directly without using the communication network 600. The electric device 200 and the communication device 500 are the same as those in the first embodiment.

[0060] Each of sensors 301-1 to 301-n has the same functions as sensor 300 in the first embodiment, and is assigned (linked) to each individual user in advance. For example, each of sensors 301-1 to 301-n may be provided in a specific area where the individual user is active, and when a vital sign is detected, it may be possible to identify that the detected vital sign belongs to that user. Also, each of sensors 301-1 to 301-n may be worn by an individual user, and when a vital sign is detected, it may be possible to identify that the detected vital sign belongs to that user.

[0061] The control device 103 controls the electrical device 200. FIG. 13 is a diagram showing an example of the internal configuration of the control device 103 shown in FIG. 12. As shown in FIG. 13, the control device 103 shown in FIG. 12 has a personal information acquisition unit 113, a learning model generation unit 120, a learning model 130, an inference unit 140, a control unit 150, and a subjective information acquisition unit 160. Note that FIG. 13 shows only the main components related to this embodiment among the components included in the control device 103 shown in FIG. 12. The learning model generation unit 120, the learning model 130, the inference unit 140, the control unit 150, and the subjective information acquisition unit 160 are the same as those in the first embodiment.

[0062] Personal information acquisition unit 113 acquires individual identification information that identifies an individual user. The individual identification information is the same as that in the first embodiment. Personal information acquisition unit 113 identifies an individual user based on identification information assigned to each of sensors 301-1 to 301-n that is transmitted from each of sensors 301-1 to 301-n. Specifically, for example, control device 103 may previously store associations between the identification information of each of sensors 301-1 to 301-n and the individual identification information, and personal information acquisition unit 113 may acquire the individual identification information associated with the identification information transmitted from sensors 301-1 to 301-n together with information indicating vital signs.

[0063] A control method in the inference phase of the control system shown in Fig. 12 will be described below. Fig. 14 is a sequence diagram for explaining an example of a control method in the inference phase of the control system shown in Fig. 12. Note that the learning model generation method in the learning phase of the control system shown in Fig. 12 may be the same as the method in the first embodiment.

[0064] First, each of sensors 301-1 to 301-n detects a vital sign of a user (step S51). The vital signs detected by each of sensors 301-1 to 301-n are as described above. Each of sensors 301-1 to 301-n transmits information indicating the detected vital sign to control device 103 (step S52). At this time, each of sensors 301-1 to 301-n transmits identification information previously assigned to the sensor to control device 103. Then, based on the identification information transmitted from sensors 301-1 to 301-n, personal information acquisition unit 113 acquires personal identification information previously associated with the identification information (step S53).

[0065] Next, when the inference unit 140 inputs the individual identifying information identified by the individual information acquisition unit 113 and the vital signs indicated by the information transmitted from the sensors 301-1 to 301-n to the learning model 130 (step S54), the inference unit 140 acquires the control content of the electric appliance 200 from the learning model 130 (step S55). Then, the control unit 150 controls the electric appliance 200 in accordance with the control content acquired by the inference unit 140 from the learning model 130 (step S56).

[0066] The reinforcement learning method in the control system shown in FIG. 12 may be the same as the method in the first embodiment.

[0067] In this manner, in this embodiment, training data is generated that associates personal identification information, vital signs of each user, and control information indicating the control content of the electrical appliance. The generated training data is used to generate a learning model that outputs control information corresponding to the personal identification information and vital signs of each user. Furthermore, the identification information of each of sensors 301-1 to 301-n is associated in advance with the personal identification information, and the personal identification information is acquired based on the identification information transmitted from each of sensors 301-1 to 301-n along with the vital signs. The acquired personal identification information and the vital signs of each user are input into the learning model, whereby control information is acquired from the learning model, and the electrical appliance is controlled using the acquired control information. This allows for the provision of an environment in which the user feels comfortable. Furthermore, after or while the user uses the provided environment, the learning model is updated (reinforcement learning) using subjective information from the user. This allows the learning model to output control information that is more suited to individual preferences.

[0068] Note that the personal identification information may be acquired by other means as long as it can acquire information that can identify the individual user. For example, if an individual can be identified using vital signs detected by a sensor, that information may be used to identify the individual. Also, an individual may be identified using fingerprint authentication or iris authentication. Also, if an individual can be identified based on date, time, day of the week, and location information, that information may be used to identify the individual.

[0069] Although the above description has been given by allocating each function (process) to each component, this allocation is not limited to the above. Furthermore, the configuration of the components is also not limited to the above-described embodiments, which are merely examples. Furthermore, each embodiment may be combined.

[0070] The processes performed by each of the control devices 100-103 may be performed by a logic circuit manufactured for each purpose. Alternatively, a computer program (hereinafter referred to as a program) describing the process procedures may be recorded on a recording medium readable by each of the control devices 100-103, and the program recorded on the recording medium may be read and executed by each of the control devices 100-103. Examples of recording media readable by each of the control devices 100-103 include removable recording media such as magneto-optical disks, DVDs (Digital Versatile Discs), CDs (Compact Discs), Blu-ray (registered trademark) Discs, and USB (Universal Serial Bus) memories, as well as memories such as ROMs (Read Only Memory), RAMs (Random Access Memory), HDDs (Hard Disc Drives), and SSDs (Solid State Drives) built into each of the control devices 100-103. The programs recorded on the recording media are read by a CPU provided in each of the control devices 100-103, and the same processes as those described above are performed under the control of the CPU. Here, the CPU operates as a computer that executes a program read from a recording medium on which the program is recorded. [Explanation of symbols]

[0071] 100, 101, 102, 103 Control device 110,111,112,113 Personal information acquisition department 120 Learning model generation unit 130 Learning Model 140 Reasoning part 150 control section 160 Subjective information acquisition unit 200 Electrical Equipment 300, 301-1 to 301-n sensors 400 Imaging device 401 Leader 402 Mike 411 Tags 500 Communication Equipment 600 Communication Network

Claims

1. A microphone that collects surrounding sounds, a personal information acquisition unit that performs a voice recognition process on the voice represented by the voice data collected by the microphone and acquires personal information that can identify the user; a vital sign sensor that detects the vital signs of the user; a control unit that controls devices that control the environment in a predetermined space; a learning model generation unit that performs machine learning on the control content of the device corresponding to the vital sign for each of the individual identifying information acquired by the personal information acquisition unit to generate a learning model; an inference unit that inputs the individual identifying information acquired by the individual information acquisition unit and the vital sign detected by the vital sign sensor into the learning model, and thereby acquires control information indicating the control content from the learning model; The microphone collects the voice when an entrance sensor detects that the user has entered an area where the device is operating; The control unit controls the device using the control content indicated by the control information acquired by the inference unit.

2. a subjective information acquisition unit that acquires subjective information from an external source; The control system according to claim 1 , wherein the learning model generation unit uses the subjective information to perform machine learning on the control content corresponding to the vital sign for each piece of personal identification information acquired by the personal information acquisition unit.

3. a communication device that receives and transmits the subjective information based on an external operation; The control system according to claim 2 , wherein the subjective information acquisition unit acquires the subjective information transmitted from the communication device.

4. 4. The control system according to claim 1, wherein the vital sign sensor detects at least one of the user's body temperature, skin temperature, respiratory rate, pulse rate, pulse variability, pulse interval, pulse cycle, heart rate, heart rate variability, heart rate interval, heart cycle, blood pressure, and facial expression as the vital sign.

5. A personal information acquisition unit that acquires personal information that can identify the user by performing voice recognition processing on the voice indicated by the voice data collected by a microphone that collects surrounding voices when an entrance sensor detects that the user has entered an area where equipment that controls the environment in a specified space is operating; a control unit that controls the device; a learning model generation unit that performs machine learning on the control content of the device corresponding to the vital sign of the user detected by the vital sign sensor for each of the personal identification information acquired by the personal information acquisition unit to generate a learning model; an inference unit that inputs the individual identifying information acquired by the individual information acquisition unit and the vital sign detected by the vital sign sensor into the learning model, and thereby acquires control information indicating the control content from the learning model; The control unit is a control device that controls the device using control content indicated by the control information acquired by the inference unit.

6. A process of performing a voice recognition process on the voice indicated by the voice data collected by a microphone that collects surrounding voices when an entrance sensor detects that a user has entered an area where equipment that controls the environment in a specified space is operating, and acquiring personal identification information that can identify the user; A process of acquiring vital signs of the user; a process of generating training data that associates the acquired personal identification information of the user, the user's vital signs, and control information that indicates control details of the device; A learning model generation method that performs machine learning using the generated training data to generate a learning model that outputs the control information according to the personal identification information and the user's vital signs.

7. A process of performing a voice recognition process on the voice indicated by the voice data collected by a microphone that collects surrounding voices when an entrance sensor detects that a user has entered an area where equipment that controls the environment in a specified space is operating, and acquiring personal identification information that can identify the user; A process of acquiring vital signs of the user; a process of inputting the acquired personal identification information and the vital signs into a learning model that has undergone machine learning to determine control content of the device that is suited to the vital signs of the user for each of the acquired personal identification information of the user, and thereby acquiring control information indicating the control content from the learning model; and controlling the device using the control content indicated by the acquired control information.

8. On the computer, a step of performing a voice recognition process on the voice indicated by the voice data collected by a microphone that collects surrounding voices when an entrance sensor detects that a user has entered an area where a device that controls the environment in a predetermined space is operating, thereby acquiring personal identification information that can identify the user; acquiring vital signs of the user; generating training data that associates the acquired personal identification information of the user, the user's vital signs, and control information that indicates control details of the device; A program for executing a procedure for generating a learning model that uses the generated training data to output control information according to the personal identification information and the user's vital signs.

9. On the computer, a step of performing a voice recognition process on the voice indicated by the voice data collected by a microphone that collects surrounding voices when an entrance sensor detects that a user has entered an area where a device that controls the environment in a predetermined space is operating, thereby acquiring personal identification information that can identify the user; acquiring vital signs of the user; a step of inputting the acquired personal identification information and the vital signs into a learning model that has undergone machine learning to determine control content of the device that is suited to the vital signs of the user for each of the acquired personal identification information of the user, thereby acquiring control information indicating the control content from the learning model; and a program for executing a procedure for controlling the device using the control content indicated by the acquired control information.

Images