Information processing method, information processing device, and computer program

Abstract

Upon detecting that a predetermined notification timing has been reached, a controlled device 20 transmits equipment information indicating the function of the controlled device 20 to an equipment control device 12. The equipment control device 12 acquires the equipment information transmitted from the controlled device 20. The equipment control device 12 acquires instruction information relating to control content of the controlled device 20. The equipment control device 12 generates an input sentence including the equipment information of the controlled device 20 and the instruction information. The equipment control device 12 inputs the input sentence into a generative AI, and acquires output information indicating control content for the controlled device 20 from the generative AI.

Description

Information processing method, information processing device, and computer program

[0001] The present disclosure relates to data processing technology, and more particularly to an information processing method, an information processing device, and a computer program.

[0002] In recent years, artificial intelligence (AI) has begun to be used to control robots and equipment. For example, Patent Document 1 below describes a method in which production parameter values ​​collected by sensors are input into an AI device, and control commands for production equipment are generated based on the analysis results of the AI ​​device.

[0003] Special Publication No. 2021-536055

[0004] A control device that controls devices using AI can be connected to a large number of devices, such as sensors and actuators. In this case, the control device needs to be configured according to each of the connected devices, and the more devices connected, the more complex the settings of the control device become. One object of the present disclosure is to provide a technology that makes it easy for a control device that controls devices using AI to follow changes in the devices connected to the control device.

[0005] In order to solve the above problem, an information processing method of one aspect of the present disclosure acquires device information, which is information transmitted from a second device that has detected that a predetermined first timing has been reached by a first device controlling the second device and which indicates the functions possessed by the second device, acquires instruction information regarding the control content of the second device, generates an input sentence including the device information and instruction information of the second device, inputs the input sentence to a generation AI, and acquires output information from the generation AI indicating the control content for the second device.

[0006] Another aspect of the present disclosure is an information processing method, in which a second device detects whether a predetermined first timing has been reached, and if it detects that the first timing has been reached, transmits device information indicating functions possessed by the second device to a first device, and obtains output information from the first device indicating control content for the second device, the output information being generated by the first device inputting an input sentence including the device information and instruction information regarding control content for the second device to a generation AI.

[0007] Yet another aspect of the present disclosure is an information processing method, in which a second device detects whether a predetermined first timing has been reached, and if the second device detects that the first timing has been reached, transmits information related to a control being executed by the second device to a first device, obtains instruction information related to the control content of the second device from the first device, generates an input sentence including device information and the instruction information of the second device, inputs the input sentence to a generation AI, and obtains output information indicating the control content for the second device from the generation AI.

[0008] Yet another aspect of the present disclosure is an information processing device including: a device information acquisition unit that acquires device information, which is information transmitted from a second device that has detected that a predetermined timing has been reached and is information indicating functions possessed by the second device; an instruction information acquisition unit that acquires instruction information regarding control content of the second device; a generation unit that generates an input sentence including the device information and the instruction information of the second device; and an output information acquisition unit that inputs the input sentence to a generation AI and acquires output information indicating control content for the second device from the generation AI.

[0009] Any combination of the above components, or any conversion of the expression of the present disclosure between a system, a computer program, a recording medium on which a computer program is recorded, etc., is also valid as an aspect of the present disclosure.

[0010] According to the technology disclosed herein, a control device that controls equipment using AI can easily follow changes in the equipment connected to the control device.

[0011] 1 is a diagram showing the configuration of a device control system of a first embodiment; FIG. 2 is a diagram showing the configuration of a device control system of a first embodiment; FIG. 3 is a block diagram showing the functional blocks of a device control device of a first embodiment; FIG. 4 is a diagram showing an example of device information; FIG. 5 is a diagram showing an example of sensor information; FIG. 6 is a diagram showing an example of spatial information; FIG. 7 is a flowchart showing the operation of a device control device of a first embodiment; FIG. 8 is a diagram showing an example of a prompt; FIG. 9 is a diagram showing an example of response information; FIG. 10 is a flowchart showing the operation of a device control device of a second embodiment; FIG. 11 is a block diagram showing the functional blocks of a device control device of a third embodiment;

[0012] The subject of the device or method of the present disclosure includes a computer. The computer executes a program to realize the functions of the subject of the device or method of the present disclosure. The computer includes, as its main hardware configuration, a processor that operates according to the program. The processor may be of any type as long as it can realize the functions by executing the program. The processor is composed of one or more electronic circuits, including a semiconductor integrated circuit (IC) or an LSI (large scale integration). While the terms IC and LSI are used here, the term may be changed depending on the degree of integration, and may also be called a system LSI, a VLSI (very large scale integration), or an ULSI (ultra large scale integration). A field programmable gate array (FPGA), which is programmed after the LSI is manufactured, or a reconfigurable logic device that can reconfigure the connections within the LSI or set up circuit partitions within the LSI, can also be used for the same purpose. Multiple electronic circuits may be integrated on a single chip or may be provided on multiple chips. Multiple chips may be integrated into a single device or may be provided on multiple devices. The program may be recorded on a non-transitory recording medium such as a computer-readable read-only memory (ROM), optical disk, or hard disk drive, or on a temporary storage medium such as a computer-readable random access memory (RAM). The program may be pre-stored on a recording medium or may be supplied to a recording medium or storage medium via a wide area communication network, including the Internet.

[0013] In recent years, the use of AI to control robots and other devices has begun. A large number of devices, such as sensors and actuators, can be connected to a control device that uses AI to control the devices. In this case, the control device must be configured to correspond to each of the connected devices, and the more devices that are connected, the more complex the control device configuration becomes.

[0014] Therefore, in the system of the embodiment (the device control system 10 described below), a generation AI is used as the AI, and the generation AI determines the control content of the device. The generation AI, also known as generative artificial intelligence, is an artificial intelligence system that generates text, images, etc. in response to a prompt. The prompt is text (in other words, a sentence) in natural language that describes the task to be performed by the generation AI. The generation AI used in the system of the embodiment uses a large-scale language model (hereinafter also referred to as "LLM" (Large Language Models)) that mimics human conversation through text-based dialogue, in other words, an artificial intelligence chatbot. Hereinafter, simply referring to the generation AI refers to the LLM. The generation AI of the embodiment outputs a text sentence (hereinafter also referred to as "answer information") in response to the input prompt.

[0015] In the system of the embodiment, the information to be set in the prompt is stored in each device connected to the control device (the controlled device 20 and the sensor 18 described below), and each device autonomously provides the information to the control device. Alternatively, the information to be set in each device may be obtained from the control device. The control device inputs a prompt containing the information provided by each device to the generation AI, and receives a response from the generation AI indicating the control content of the device. This improves the ease with which the control device can follow changes in the devices connected to the control device (in other words, the ease with which the prompt can follow).

[0016] <First embodiment> Fig. 1 shows the configuration of a device control system 10 according to a first embodiment. The device control system 10 includes a plurality of controlled devices 20 installed in a target space. The target space in Fig. 1 is the interior space of a vehicle (in other words, the vehicle compartment). The plurality of controlled devices 20 include electronic devices, electrical appliances, and various actuators that are controlled using generation AI. In the device control system 10 of Fig. 1, the plurality of controlled devices 20 include an air conditioner 21, power windows 22, interior lights 23, audio equipment 24, and a display device 25.

[0017] When each of the plurality of controlled devices 20 detects that a predetermined notification timing has arrived, it transmits device information indicating the functions of the device itself (e.g., air conditioning, window opening / closing, lighting, etc.) to the device control device 12. The notification timing may include at least one of the following: the power of the controlled device 20 is turned on; the controlled device 20 is connected to a communication network; the controlled device 20 is connected to other devices including the device control device 12 or other controlled devices 20; the time when the state of the controlled device 20 changes; and the lapse of a predetermined time (e.g., 10 minutes) since the previous notification.

[0018] The device control system 10 also includes a device control device 12, an input device 14, a spatial information storage device 16, and a sensor 18. The devices shown in Fig. 1 are connected via a communication network including a LAN, a WAN, the Internet, etc.

[0019] The device control device 12 is an information processing device that utilizes generation AI to control at least one controlled device 20 in a manner that responds to instruction information from a user. The device control device 12 may be a computer, a tablet terminal, or a smartphone on which a device control application is installed.

[0020] The input device 14 is an information processing device into which a user inputs instruction information. The input device 14 may be a computer, a tablet terminal, or a smartphone. The input device 14 transmits instruction information regarding the control content of the controlled device 20 to the appliance control device 12 in response to a user's operation. The instruction information regarding the control content of the controlled device 20 may include information indicating an operation mode of the controlled device 20 desired by the user, and may also include information indicating an item to be achieved by the function of the controlled device 20. The instruction information may include an item to be achieved by the function of the air conditioner 21 or the power window 22, such as "make it comfortable" or "make it cool." The instruction information may also include an item to be achieved by the function of the interior light 23, such as "make it brighter" or "make it darker."

[0021] The spatial information storage device 16 is an information processing device that stores information (hereinafter also referred to as "spatial information") about a predetermined target space in which the controlled device 20 operates. The spatial information may include the type of space (living room, bedroom, conference room, car cabin, etc.), information about the structure of the target space (shape, presence or absence of windows, etc.), and information about devices installed in the target space (sensors, controlled devices, etc.). The functions of at least two devices among the device control device 12, the input device 14, and the spatial information storage device 16 may be implemented in a single information processing device.

[0022] The sensor 18 measures a predetermined physical quantity in the target space. Although one sensor 18 is depicted in Fig. 1, the equipment control system 10 may include multiple sensors 18 measuring the same type of physical quantity or different types of physical quantities. The physical quantity measured by the sensor 18 may include, for example, at least one of temperature, humidity, light, color, carbon dioxide, pressure, magnetism, speed, and acceleration.

[0023] When the sensor 18 detects that a predetermined notification timing has arrived, it transmits sensor information, which is information about the surrounding situation of the controlled device 20, to the equipment control device 12. The sensor information can be said to be information about a physical quantity measured by the sensor 18, or information indicating the situation of the target space in which the controlled device 20 operates (in other words, the environment). The notification timing may include at least one of the following: the sensor 18 being powered on; the sensor 18 being connected to a communication network; the sensor 18 being connected to other devices including the equipment control device 12 and multiple controlled devices 20; the time when the sensor value has changed to a predetermined amount; and the elapse of a predetermined time (e.g., 10 minutes) since the previous notification.

[0024] Fig. 2 also shows the configuration of the device control system 10 of the first embodiment. The device control system 10 of Fig. 2 differs from the device control system 10 of Fig. 1 in that the target space in which the controlled devices 20 operate is a room in a building. The device control system 10 of Fig. 2 includes a plurality of controlled devices 20 installed in the room. In Fig. 2, the plurality of controlled devices 20 includes an air conditioner 21, an audio device 24, a display device 25, an electric fan 26, a light 27, and a microwave oven 28.

[0025] FIG. 3 is a block diagram showing functional blocks of the device control device 12 according to the first embodiment. The device control device 12 includes a computer, which executes programs to realize the various functions shown in FIG. 3 . The computer includes hardware such as a memory into which the programs are loaded, one or more processors that execute the loaded programs, an auxiliary storage device, and other LSIs. The processor is composed of multiple electronic circuits, including semiconductor integrated circuits and LSIs, and the multiple electronic circuits may be mounted on a single chip or multiple chips. The functional blocks shown in FIG. 3 are realized by cooperation between hardware and software. Therefore, it will be understood by those skilled in the art that these functional blocks can be realized in various ways, using only hardware, only software, or a combination thereof.

[0026] The device control device 12 includes a processing unit 30, a storage unit 32, and a communication unit 34. The processing unit 30 is realized by a processor (such as a CPU) of the device control device 12 and executes various information processes. The storage unit 32 is realized by a memory or storage of the device control device 12 and stores data referenced or updated by the processing unit 30. The communication unit 34 communicates with external devices in accordance with a predetermined communication protocol. The processing unit 30 transmits and receives data to and from the input device 14, the spatial information storage device 16, the sensor 18, and the controlled device 20 via the communication unit 34.

[0027] The storage unit 32 includes an LLM storage unit 40 and a prompt storage unit 42. The LLM storage unit 40 stores data of the generated AI (LLM). The prompt storage unit 42 stores prompts to be input to the generated AI.

[0028] The processing unit 30 includes a device information acquisition unit 50, a spatial information acquisition unit 52, an instruction information acquisition unit 54, a prompt generation unit 56, a response acquisition unit 58, and a device control unit 60. The functions of these multiple functional blocks are implemented in a computer program, and this computer program may be installed in the storage of the device control device 12. The processor of the device control device 12 may perform the functions of these multiple functional blocks by reading this computer program into main memory and executing it.

[0029] The device information acquiring unit 50 acquires device information transmitted from each of the plurality of controlled devices 20. The prompt generating unit 56 sequentially adds the device information acquired by the device information acquiring unit 50 to the prompt data stored in the prompt storage unit 42.

[0030] The spatial information acquisition unit 52 acquires spatial information related to the target space in which the controlled device 20 operates, which is stored in the spatial information storage device 16, from the spatial information storage device 16. The spatial information acquisition unit 52 may periodically acquire the spatial information from the spatial information storage device 16. Alternatively, the spatial information storage device 16 may periodically transmit the spatial information to the device control device 12. The prompt generation unit 56 sequentially adds the spatial information acquired by the spatial information acquisition unit 52 to the prompt data stored in the prompt storage unit 42.

[0031] The instruction information acquisition unit 54 acquires instruction information related to the control content of the controlled device 20, transmitted from the input device 14. The prompt generation unit 56 sequentially adds the instruction information acquired by the instruction information acquisition unit 54 to the prompt data stored in the prompt storage unit 42. In this way, the prompt generation unit 56 generates a prompt including device information provided from one or more controlled devices 20, sensor information provided from the sensor 18, spatial information provided from the spatial information storage device 16, and instruction information provided from the input device 14.

[0032] The answer acquisition unit 58 inputs the prompt stored in the prompt storage unit 42 to the generated AI stored in the LLM storage unit 40. The answer acquisition unit 58 acquires answer information, which is information output by the generated AI in response to the prompt and indicates the control content for at least one controlled device 20 among the multiple controlled devices 20.

[0033] The device control unit 60 controls the controlled device 20 based on the response information acquired by the response acquisition unit 58. The response information may be in a predetermined format, and the device control unit 60 may extract the controlled device and the control content from the response information based on that format. The device control unit 60 may transmit a control signal in a predetermined format indicating the control content extracted from the response information to the controlled device (at least one controlled device 20) extracted from the response information. Note that the device control unit 60 may transmit an instruction indicating the control content extracted from the response information to the controlled device (at least one controlled device 20) as an instruction for the controlled device. In this case, the controlled device itself is the entity that controls the controlled device.

[0034] The operation of the device control system 10 configured as described above will be described. Each of the multiple controlled devices 20 operating in the target space (here, the vehicle interior) transmits device information of the device itself to the device control device 12 every time it detects that a predetermined notification timing has arrived. Note that the device information may be the latest device information including the latest control parameter values, or may be device information including control parameter values ​​that are not the latest.

[0035] The sensor 18, which measures a predetermined physical quantity in the vehicle interior space, transmits sensor information indicating the current state of the vehicle interior space to the device control device 12 each time it detects that a predetermined notification timing has arrived. This sensor information includes a current sensor value (measurement value). The space information storage device 16 transmits space information regarding the vehicle interior space to the device control device 12 each time it detects that a predetermined notification timing has arrived.

[0036] FIG. 4 shows an example of device information. The device information includes an identifier, a function description 1, location information 1, possible control parameter values ​​1, and a current control parameter value 1. The identifier is identification information of the controlled device 20. The function description 1 is a description of the functions possessed by the controlled device 20. The location information 1 is information about the location where the controlled device 20 is installed, for example, information indicating the installation location of the controlled device 20 in the target space. The possible control parameter values ​​1 are information about values ​​with which the controlled device 20 can be controlled, specifically, information indicating the type and range of control parameter values ​​that can be set for the controlled device 20. The current control parameter value 1 is information about the current control state of the controlled device, specifically, the value of the control parameter currently set for the controlled device 20. FIG. 4 illustrates an example of device information values ​​for an air conditioner 21. If the controlled device 20 has multiple functions, values ​​are also set for function description 2 and subsequent functions. In addition, the device information may include operating language information (information about a programming language or machine language) and operating specifications for the controlled device 20. The above-mentioned function description, possible values ​​of control parameters, operation language information, and information about operation standards can also be said to be information about controls that can be executed by controlled device 20 .

[0037] FIG. 5 shows an example of sensor information. The sensor information includes an identifier, a function description 1, location information 1, a possible sensor value 1, and a current sensor value. The identifier is identification information of the sensor 18. The function description 1 is a description of the function of the sensor 18 (here, the type of physical quantity to be measured). The location information 1 is information about the location where the sensor 18 is installed, for example, information indicating the installation location of the sensor 18 within the target space. The possible sensor value 1 is information indicating the range of values ​​output by the sensor 18, in other words, the range of physical quantities that the sensor 18 can measure. The current sensor value 1 is the current measurement value. The sensor information in FIG. 5 shows sensor information for the sensor 18, which includes a temperature sensor and a humidity sensor. Items marked with "1" are information about the temperature sensor, and items marked with "2" are information about the humidity sensor.

[0038] FIG. 6 shows an example of spatial information. The spatial information includes an identifier, a function description, and a structure description. The identifier is identification information for the target space. The function description is information indicating the type and use of the target space. The structure description is information indicating the structure of the target space (such as the shape of the space and whether or not it has windows).

[0039] 7 is a flowchart showing the operation of the device control device 12 of the first embodiment. The device control device 12 of the first embodiment repeatedly executes the process shown in FIG.

[0040] The device information acquisition unit 50 of the device control device 12 acquires device information transmitted from each of the multiple controlled devices 20. Each time device information is acquired (Y in S10), the prompt generation unit 56 of the device control device 12 sequentially adds the acquired device information to the prompt data stored in the prompt storage unit 42 (S11). If device information has not been acquired (N in S10), the process of S11 is skipped.

[0041] The device information acquisition unit 50 of the device control device 12 also acquires sensor information transmitted from the sensor 18. Each time sensor information is acquired (Y in S12), the prompt generation unit 56 of the device control device 12 sequentially adds the acquired sensor information to the prompt data stored in the prompt storage unit 42 (S13). If sensor information has not been acquired (N in S12), the process of S13 is skipped.

[0042] The spatial information acquisition unit 52 of the device control device 12 acquires the spatial information transmitted from the spatial information storage device 16. Each time spatial information is acquired (Y in S14), the prompt generation unit 56 of the device control device 12 sequentially adds the acquired spatial information to the prompt data stored in the prompt storage unit 42 (S15). If spatial information has not been acquired (N in S14), the process of S15 is skipped.

[0043] The user inputs instruction information to the input device 14. For example, the input device 14 may display multiple options indicating different instruction information on a screen, and the user may select desired instruction information from the multiple options. The user may also input instruction information to the input device 14 by voice or by other means. The input device 14 transmits the instruction information input or selected by the user to the appliance control device 12.

[0044] The instruction information acquisition unit 54 of the device control device 12 acquires the instruction information transmitted from the input device 14. When the instruction information is acquired (Y in S16), the prompt generation unit 56 of the device control device 12 adds the acquired instruction information to the prompt data stored in the prompt storage unit 42 (S17). In response to the acquisition of the instruction information, the answer acquisition unit 58 of the device control device 12 inputs the prompt stored in the prompt storage unit 42, to which the instruction information has been added, into the generation AI stored in the LLM storage unit 40. The answer acquisition unit 58 acquires the answer information output from the generation AI in response to the prompt (S18).

[0045] FIG. 8 shows an example of a prompt. The prompt includes sensor information 100, device information 101, spatial information 102, supplemental information 103, and instruction information 104. The sensor information 100 includes text 110 to text 114. Text 110 indicates an identifier and location information. Text 111 indicates function descriptions 1, 2, and 3 (types of physical quantities to be measured, specifically, carbon dioxide, temperature, and humidity). Text 112 indicates current sensor value 1 (carbon dioxide concentration). Text 113 indicates current sensor value 2 (temperature). Text 114 indicates current sensor value 3 (humidity).

[0046] The device information 101 includes text 120 to text 123. Text 120 indicates location information. Text 121 indicates a function description (fan). Text 122 indicates possible values ​​of the control parameter value (stop, weak wind, medium wind, strong wind). Text 123 indicates the current control parameter value (stop). The supplemental information 103 indicates a sentence explaining the format of the prompt. Specifically, the supplemental information 103 in FIG. 8 indicates a method for identifying the controlled device and a location where the control parameter values ​​that can be set for the controlled device are described.

[0047] 9 shows an example of response information. The response information in FIG. 9 is an example of free-format text. Specifically, the response information in FIG. 9 indicates that the generation AI selected the fan on the ceiling of the driver's seat as the device to be controlled and recommended that it be operated at medium wind speed as the control content.

[0048] Returning to FIG. 7 , the equipment control unit 60 of the equipment control device 12 controls the controlled device 20 based on the response information of the generation AI (S19). For example, if the response information is as shown in FIG. 9 , the equipment control unit 60 may transmit a control signal to the fan, which is the controlled device 20, instructing it to operate in moderate wind conditions. The equipment control unit 60 may extract the controlled device and control content recommended by the generation AI from the response information by analyzing the free-format response information using known natural language processing. Furthermore, if the response information is in a predetermined format, the equipment control unit 60 may extract the controlled device and control content recommended by the generation AI, which are written in a predetermined position in the response information. If the instruction information has not been acquired (N in S16), the processing of S17 to S19 is skipped.

[0049] According to the device control system 10 of the first embodiment, the latest device information of the controlled device 20 and the latest sensor information of the sensor 18 are sequentially added to the prompt input to the generating AI. This improves the ease with which the device control device 12 and the prompt can follow changes in the controlled device 20 and the sensor 18 connected to the device control device 12 when the controlled device 20 is controlled using the generating AI. As a result, it becomes easier to obtain a response from the generating AI that is adapted to changes in the controlled device 20 and the sensor 18 connected to the device control device 12. Furthermore, according to the device control system 10, by including spatial information in which the controlled device 20 operates in the prompt, it becomes easier to obtain a response from the generating AI that is adapted to the space in which the controlled device 20 operates.

[0050] A first modified example of the first embodiment will be described. In the device control system 10 of the first embodiment, the device control device 12 is provided with the generating AI (LLM). As a modified example, a device (herein referred to as an AI device) separate from the device control device 12 may be provided with the generating AI (LLM). In this case, the answer acquisition unit 58 of the device control device 12 may send a prompt to the AI ​​device. The AI ​​device may generate answer information by inputting a prompt to the generating AI, and send the generated answer information to the device control device 12.

[0051] A second modification of the first embodiment will be described. In the first embodiment, the prompt generation unit 56 of the device control device 12 generated a prompt including sensor information, device information, spatial information, and instruction information. As a modification, the prompt includes device information and instruction information as minimum items, but may not include sensor information and spatial information. Furthermore, instruction information may be set in the prompt by a method other than input or selection by the user. For example, the prompt generation unit 56 may automatically generate one or more prompts using each of one or more predetermined instruction information.

[0052] A third modified example of the first embodiment will be described. In the first embodiment, the prompt generation unit 56 of the device control device 12 sequentially adds acquired sensor information or device information to the prompt. As a modified example, if existing sensor information or device information with the same identifier as the acquired sensor information or device information has already been set in the prompt, the prompt generation unit 56 may replace the existing sensor information or device information with the acquired sensor information or device information. In other words, the prompt generation unit 56 may update the device information of the controlled device 20 and the sensor information of the sensor 18 recorded in the prompt on a device-by-device basis.

[0053] A fourth modified example of the first embodiment will be described. In the first embodiment, the device control device 12 includes a device control unit 60 that controls the controlled device 20 based on the answer information. As a modified example, the device control device 12 may include an answer information output unit instead of the device control unit 60. The answer information output unit may display the answer information of the generated AI on a predetermined display device, or may output a sound indicating the answer information from a predetermined speaker. The user may adjust the operating mode of the controlled device 20 according to the answer information displayed on the display device or output from the speaker.

[0054] A fifth modification of the first embodiment will be described. Although not mentioned in the first embodiment, the device information, sensor information, and space information set in the prompt may each include a timestamp. This timestamp may indicate the date and time when the device information, etc. was generated by the information source (such as the controlled device 20), the date and time when the device information, etc. was received by the device control device 12, or the date and time when the device information, etc. was set in the prompt. The prompt generation unit 56 of the device control device 12 may delete from the prompt any device information, sensor information, or space information whose timestamp date and time is more than a predetermined time away from the current date and time. Furthermore, the generation AI may be configured to select a device to be controlled and control content by assigning a higher weight to device information, etc. whose timestamp value is closer to the current date and time.

[0055] Second Embodiment In the second embodiment, the content already explained in the first embodiment will be omitted as appropriate, and differences from the first embodiment will be mainly explained. In the following explanation, components of the second embodiment that are the same as or correspond to components of the first embodiment will be denoted by the same reference numerals as in the first embodiment.

[0056] The devices constituting the device control system 10 of the second embodiment are the same as the devices constituting the device control system 10 of the first embodiment shown in Fig. 1. The controlled devices 20 and sensors 18 can be deleted from the device control system 10 when they are discarded or removed. The device control device 12 of the second embodiment differs from the device control device 12 of the first embodiment in that the deletion of the controlled devices 20 and sensors 18 from the device control system 10 is reflected in the prompt.

[0057] In the second embodiment, each of the multiple controlled devices 20 transmits predetermined deletion information to the device control device 12 when it detects that its own device will transition to a state uncontrollable by the external device. For example, when a shutdown instruction is input, the controlled device 20 may detect that its own device will transition to a state uncontrollable by the external device and transmit the deletion information before shutting down. Furthermore, when a communication disconnection instruction is input, the controlled device 20 may detect that its own device will transition to a state uncontrollable by the external device and transmit the deletion information before disconnecting the communication. The deletion information may include an identifier of the controlled device, may be information indicating that its own device will transition to a state uncontrollable by the external device, or may be information instructing deletion of device information of the controlled device.

[0058] The device information acquisition unit 50 of the device control device 12 acquires deletion information transmitted from each of the multiple controlled devices 20. When the prompt generation unit 56 of the device control device 12 acquires deletion information transmitted from a certain controlled device 20, it excludes the device information of the controlled device 20 from the prompt stored in the prompt storage unit 42. For example, the prompt generation unit 56 may delete, from one or more pieces of device information recorded in the prompt, device information having the same identifier as the identifier of the controlled device 20 indicated in the deletion information.

[0059] In the second embodiment, when the sensor 18 detects that the sensor 18 is transitioning to a state in which it is unable to output sensor information, the sensor 18 transmits predetermined deletion information to the device control device 12. For example, when a shutdown instruction is input, the sensor 18 may detect that the sensor 18 is transitioning to a state in which it is unable to output sensor information and transmit the deletion information before the shutdown is executed. Furthermore, when a communication disconnection instruction is input, the sensor 18 may detect that the sensor 18 is transitioning to a state in which it is unable to output sensor information and transmit the deletion information before the communication disconnection is executed. The deletion information may include an identifier of the sensor 18, may be information indicating that the sensor 18 is transitioning to a state in which it is unable to output sensor information, or may be information instructing the deletion of the device information of the sensor 18.

[0060] The device information acquisition unit 50 of the device control device 12 acquires the deletion information transmitted from the sensor 18. When the prompt generation unit 56 of the device control device 12 acquires the deletion information transmitted from the sensor 18, it excludes the sensor information of the sensor 18 from the prompt stored in the prompt storage unit 42. For example, the prompt generation unit 56 may delete, from one or more pieces of sensor information recorded in the prompt, device information having the same identifier as the identifier of the sensor 18 indicated in the deletion information.

[0061] FIG. 10 is a flowchart showing the operation of the device control device 12 of the second embodiment. The processes of S20 to S23 in FIG. 10 are the same as the processes of S10 to S13 in FIG. 7 described in the first embodiment, and therefore will not be described again. The device information acquisition unit 50 of the device control device 12 receives deletion information transmitted from the controlled device 20 or the sensor 18 (Y in S24). The prompt generation unit 56 of the device control device 12 deletes from the prompt the device information or sensor information for which the same identifier as the identifier indicated in the deletion information is set (S25). If the deletion information is not received (N in S24), the process of S25 is skipped. The processes from S26 onward in FIG. 10 are the same as the processes from S14 onward in FIG. 7 described in the first embodiment, and therefore will not be described again.

[0062] The device control system 10 of the second embodiment can further improve the ease with which the device control device 12 and the prompt can follow changes in the controlled devices 20 and sensors 18 connected to the device control device 12. As a result, it becomes easier to obtain answers from the generating AI that are more adapted to changes in the controlled devices 20 and sensors 18 connected to the device control device 12.

[0063] Third Embodiment In the second embodiment, the content already explained in the first embodiment will be omitted as appropriate, and differences from the first embodiment will be mainly explained. In the following explanation, components of the third embodiment that are the same as or correspond to components of the first embodiment will be denoted by the same reference numerals as in the first embodiment.

[0064] The devices constituting the device control system 10 of the third embodiment are the same as the devices constituting the device control system 10 of the first embodiment shown in Fig. 1. The device control device 12 of the third embodiment differs from the device control device 12 of the first embodiment in the method of generating a prompt.

[0065] 11 is a block diagram showing functional blocks of the device control device 12 of the third embodiment. The device control device 12 of the third embodiment differs from the device control device 12 of the first embodiment in that it includes a sensor information storage unit 44, a device information storage unit 46, and a spatial information storage unit 48 instead of the prompt storage unit 42.

[0066] The device information acquiring unit 50 sequentially stores the sensor information transmitted from the sensor 18 in the sensor information storage unit 44. The device information acquiring unit 50 also sequentially stores the device information transmitted from each of the plurality of controlled devices 20 in the device information storage unit 46. The space information acquiring unit 52 stores the space information transmitted from the space information storage device 16 in the space information storage unit 48.

[0067] When the instruction information acquisition unit 54 receives instruction information transmitted from the input device 14, the prompt generation unit 56 generates a prompt including one or more pieces of sensor information stored in the sensor information storage unit 44, one or more pieces of device information stored in the device information storage unit 46, one or more pieces of space information stored in the space information storage unit 48, and the instruction information. The answer acquisition unit 58 inputs the prompt generated by the prompt generation unit 56 to the generation AI to acquire answer information.

[0068] The device control system 10 of the third embodiment also achieves the same effects as the device control system 10 of the first embodiment. That is, it is possible to improve the ease with which the device control device 12 and the prompt can follow changes in the controlled devices 20 and sensors 18 connected to the device control device 12.

[0069] The device control system 10 of the second embodiment and the device control system 10 of the third embodiment can also be combined. In this case, when the device information acquisition unit 50 of the device control device 12 receives deletion information from the controlled device 20, it may delete device information having the same identifier as the identifier indicated in the deletion information from the device information stored in the device information storage unit 46. Furthermore, when the device information acquisition unit 50 receives deletion information from the sensor 18, it may delete sensor information having the same identifier as the identifier indicated in the deletion information from the sensor information storage unit 44. This allows the deletion of the controlled device 20 and the sensor 18 from the device control system 10 to be reflected in the prompt.

[0070] The present disclosure has been described above based on the embodiments. The embodiments are merely examples, and it will be understood by those skilled in the art that various modifications are possible in the combination of each component or each treatment process of the embodiments, and that such modifications are also within the scope of the present disclosure.

[0071] Modifications of each embodiment will be described. In each embodiment, each of the sensor 18 and the controlled device 20 holds sensor information (such as a description of the function, location information, and possible control parameter values) or device information indicating the functions of the device itself. Hereinafter, both sensor information and device information will be collectively referred to as "device information." As a modification, each of the sensor 18 and the controlled device 20 may not hold device information. Instead, the device information may be registered in the storage unit 32 included in the device control device 12 or in a storage device (not shown) communicably connected to the device control device 12 via a network.

[0072] In this modification, the sensor 18 and the controlled device 20 may hold information indicating their own type or identification information for identifying each of them, instead of their respective device information. Then, when a predetermined timing arrives, the device control device 12 may acquire the device information of the corresponding sensor 18 or controlled device 20 from the storage unit 32 or a storage device based on the identification information, etc., of the sensor 18 or controlled device 20. The storage device here is, for example, a server, and the server may store device information corresponding to each piece of identification information, etc., or may store device information corresponding to information indicating the type of the sensor 18 or controlled device 20.

[0073] Another modified example of each embodiment will be described. The controlled device 20 may be provided with a generating AI or may be connected to the generating AI via a network. The controlled device 20 may determine the control content of the controlled device 20 (i.e., its own device) using the generating AI, similar to the device control device 12 of the embodiment. In this modified example, the device control device 12 may be provided with a generating AI, similar to the embodiment, or may not be provided with a generating AI.

[0074] In this modification, when a predetermined timing arrives, the controlled device 20 outputs information about the control being performed by the controlled device 20 to the device control device 12 together with or instead of the device information. The information about the control may include information about the purpose and / or goal of the control. Information about the control being performed by each controlled device 20, output from the multiple controlled devices 20, is successively added or updated to the storage unit 32 of the device control device 12. Note that the storage unit 32 of the device control device 12 may also store information output from each sensor 18.

[0075] When the equipment control device 12 is equipped with a generation AI, the equipment control device 12 may output a control signal to the specific controlled device 20 based on the equipment information and instruction information output from the controlled device 20 (herein referred to as a specific controlled device 20). At the same time, the equipment control device 12 also outputs information about the control being performed by the other multiple controlled devices 20 (including information about the purpose and goal of the control, as described above) to the specific controlled device 20. The equipment control device 12 may further output equipment information about the other multiple controlled devices 20 to the specific controlled device 20. Note that when the equipment control device 12 is referred to as a first device and the specific controlled device 20 is referred to as a second device, the other multiple controlled devices 20 can be said to be a third device.

[0076] On the other hand, if the device control device 12 does not have a generation AI, the device control device 12 outputs instruction information and information (which may further include device information) related to the control being performed by the other multiple controlled devices 20 to the specific controlled device 20. The specific controlled device 20 generates an input sentence using the device information of its own device, the instruction information provided by the device control device 12, and information (which may further include device information) related to the control of the other multiple controlled devices 20, inputs the generated input sentence to the generation AI, and obtains output information related to the control content of the specific controlled device 20 itself from the generation AI. In other words, the specific controlled device 20 generates a control signal for its own device by inputting the input sentence to the generation AI.

[0077] According to the configuration of this modified example, when a specific controlled device 20 is equipped with a generation AI, the specific controlled device 20 can obtain, when a predetermined timing arrives, equipment information of multiple other controlled devices 20 and information regarding the control being performed in the multiple other controlled devices 20 from the equipment control device 12, thereby enabling more appropriate control of the device itself based on the obtained information.

[0078] For example, when a predetermined timing arrives, if the air conditioner 21 acquires information through the equipment control device 12 that the power window 22 is fully open, it can determine the control content of the device itself (for example, to stop the fan) based on the information.

[0079] Furthermore, for example, when the timing arrives by turning on the power of the electric fan 26 or by connecting the electric fan 26 to a communication network, the electric fan 26 outputs information about the control being performed by the electric fan 26 to the equipment control device 12. Then, when the timing arrives for the air conditioner 21, the air conditioner 21 can acquire the information about the electric fan 26 and determine the control content of the air conditioner 21 (for example, setting the fan to "low") based on the information. In this way, the configuration of this modified example makes it possible to realize cooperative control between controlled devices 20 each having a generation AI.

[0080] In this modification, the predetermined timing may include, in addition to the timing already described in the embodiment, the time when information is output from any of the controlled devices 20 to the equipment control device 12, and may also include the time when information is output from the equipment control device 12 to any of the controlled devices 20. This makes it possible to realize real-time cooperative control between the controlled devices 20.

[0081] Any combination of the above-described embodiments and modifications is also useful as an embodiment of the present disclosure. A new embodiment resulting from a combination combines the effects of the combined embodiments and modifications. It will also be understood by those skilled in the art that the functions to be performed by each component recited in the claims can be realized by each component shown in the embodiments and modifications, either alone or in combination.

[0082] Each device in the above embodiments and modifications is specifically a computer system comprising a microprocessor, ROM, RAM, hard disk unit, display unit, keyboard, mouse, etc. A computer program is recorded on the RAM or hard disk unit. Each device achieves its function when the microprocessor operates in accordance with the computer program. Here, the computer program is composed of a combination of multiple instruction codes that indicate commands to the computer to achieve a predetermined function.

[0083] In each of the above embodiments and variations, some or all of the constituent elements of each device may be configured from a single system LSI (Large Scale Integration). A system LSI is an ultra-multifunctional LSI manufactured by integrating multiple components on a single chip, and specifically, is a computer system configured to include a microprocessor, ROM, RAM, etc. A computer program is stored in the RAM. The system LSI achieves its functions when the microprocessor operates in accordance with the computer program.

[0084] Although the term "system LSI" is used here, it may also be referred to as an IC, LSI, super LSI, or ultra LSI depending on the level of integration. Furthermore, the integrated circuit implementation is not limited to LSIs; it may also be implemented using dedicated circuits or general-purpose processors. FPGAs (Field Programmable Gate Arrays), which can be programmed after LSI fabrication, or reconfigurable processors, which allow the connections and settings of circuit cells within the LSI to be reconfigured, may also be used. Furthermore, each component constituting each of the above devices may be integrated individually on a single chip, or may be integrated into a single chip that includes some or all of the components. Furthermore, if an integrated circuit technology that replaces LSIs emerges due to advances in semiconductor technology or other derivative technologies, that technology may naturally be used to integrate functional blocks. Biotechnology, for example, is also a possibility.

[0085] Furthermore, some or all of the components constituting each of the above devices may be configured as an IC card or a standalone module that can be attached to each device. The IC card or module is a computer system composed of a microprocessor, ROM, RAM, etc. The IC card or module may include the above-mentioned ultra-multifunctional LSI. The IC card or module achieves its functions when the microprocessor operates in accordance with a computer program. The IC card or module may be tamper-resistant.

[0086] The present disclosure may be the above-described methods. It may also be a computer program for implementing these methods on a computer, or a digital signal comprising the computer program. The present disclosure may also be a computer program or a digital signal recorded on a computer-readable recording medium, such as a flexible disk, hard disk, CD-ROM, MO, DVD, DVD-ROM, DVD-RAM, BD (Blu-ray (registered trademark) Disc), or semiconductor memory. It may also be a digital signal recorded on such a recording medium. The present disclosure may also be a computer program or a digital signal transmitted via a telecommunications line, a wireless or wired communication line, a network such as the Internet, data broadcasting, or the like.

[0087] The present disclosure may also be a computer system having a microprocessor and a memory, the memory storing the computer program, and the microprocessor operating in accordance with the computer program. The present disclosure may also be implemented by another independent computer system by recording the program or the digital signal on the recording medium and transferring it, or by transferring the program or the digital signal via the network or the like.

[0088] <Supplementary Notes> The above embodiments and variations disclose the following technologies. [Technology 1] An information processing method in which a first device controlling a second device: acquires device information, the device information being information transmitted from the second device that has detected that a predetermined first timing has been reached and indicating the functions of the second device; acquires instruction information relating to the control content of the second device; generates an input sentence including the device information of the second device and the instruction information; inputs the input sentence to a generation AI; and acquires output information indicating the control content for the second device from the generation AI. This information processing method makes it possible to utilize the generation AI to acquire output information indicating the control content for the second device and improves the ease with which the first device (in other words, the input sentence to the generation AI) can follow changes in the second device. [Technology 2] The information processing method described in Technology 1, in which the first device controls the second device based on the output information. This information processing method makes it possible to automatically control the second device in accordance with the response of the generation AI. [Technology 3] The information processing method according to Technology 1 or 2, in which the first device acquires sensor information, which is information about the surrounding situation of the second device and is transmitted from a sensor device that has detected that a predetermined second timing has been reached, and the first device generates an input sentence further including the sensor information. This information processing method can improve the ease with which the first device (in other words, the input sentence to the generation AI) can follow changes in the sensor device connected to the first device. [Technology 4] The information processing method according to Technology 3, in which the second timing includes at least one of the following: when the sensor device is powered on, when the sensor device is connected to a communication network, when the sensor information indicates a predetermined change, and when a predetermined time has elapsed since the previous second timing. This information processing method can acquire output information indicating control details for the second device at an appropriate timing. [Technology 5] The information processing method according to any one of Technology 1 to 4, in which the first device further acquires spatial information about the space in which the second device operates, and the first device generates an input sentence further including the spatial information.This information processing method makes it easier to obtain a response from the generation AI that is compatible with the space in which the second device is operating. [Technology 6] The information processing method described in any one of Techniques 1 to 5, in which the first device acquires predetermined deletion information transmitted from the second device that has detected that its own device is transitioning to an uncontrollable state, and, upon acquiring the deletion information transmitted from the second device, the first device excludes device information of the second device from the input sentence. This information processing method further improves the ease with which the first device (in other words, the input sentence to the generation AI) can follow changes in the second device connected to the first device. [Technology 7] The information processing method, in which the second device detects whether a predetermined first timing has been reached, and, upon detecting that the first timing has been reached, transmits device information indicating the functions possessed by the first device to the first device, and acquires output information from the first device that indicates the control content for the second device, the output information being generated by the first device inputting an input sentence to the generation AI from the first device, the input sentence including the device information and instruction information regarding the control content of the second device. According to this information processing method, the second device can obtain output information from the first device indicating control details for the second device that follow changes in the first device. [Technology 8] The information processing method described in any one of Techniques 1 to 7, wherein the device information includes information about the location where the second device is installed. According to this information processing method, the installation location of the second device is included in the input sentence, making it easier to obtain output information indicating control details that take the installation location of the second device into account. [Technology 9] The information processing method described in any one of Techniques 1 to 8, wherein the device information includes information about controls that the second device can execute. According to this information processing method, information about controls that the second device can execute (e.g., values ​​that can control the second device) is included in the input sentence, making it easier to obtain output information indicating control details that can be applied to the second device. [Technology 10] The information processing method described in any one of Techniques 1 to 9, wherein the device information includes information about the current control state of the second device. According to this information processing method, the current control state of the second device is included in the input sentence, making it easier to obtain output information indicating control details that take the current control state of the second device into account.[Technology 11] The information processing method according to any one of Techniques 1 to 10, wherein the first timing includes at least one of the following: when the second device is powered on, when the second device is connected to a communication network, when the second device is connected to the first device, when the state of the second device changes, and when a predetermined time has elapsed since the previous first timing. This information processing method allows output information indicating control details for the second device to be obtained at an appropriate time. [Technology 12] The information processing method includes: a second device detects whether a predetermined first timing has been reached; if it detects that the first timing has been reached, transmits information regarding the control being performed by the second device to the first device; acquires instruction information regarding the control details of the second device from the first device; generates an input sentence including the instruction information and device information indicating functions possessed by the second device; inputs the input sentence to a generation AI; and acquires output information indicating the control details for the second device from the generation AI. According to this information processing method, output information indicating the control content for the second device can be obtained by utilizing the generation AI, and the ease with which input sentences to the generation AI can follow changes in the second device can be improved. [Technology 13] The information processing method described in Technology 12 further obtains information regarding the control of a third device different from the second device from the first device, and generates an input sentence including the device information of the second device, the instruction information, and information regarding the control of the third device. According to this information processing method, the second device can obtain information regarding the control of the third device from the first device, and can more appropriately control its own device based on the obtained information. [Technology 14] An information processing device comprising: a device information acquisition unit that acquires device information, which is information transmitted from a second device that has detected that a predetermined timing has been reached and is information indicating functions possessed by the second device; an instruction information acquisition unit that acquires instruction information regarding control content of the second device; a generation unit that generates an input sentence including the device information and the instruction information of the second device; and an output information acquisition unit that inputs the input sentence to a generation AI and acquires output information from the generation AI that indicates control content for the second device.This information processing device can utilize a generation AI to obtain output information indicating control details for a second device, and can improve the ease with which the device (in other words, input sentences to the generation AI) can follow changes in the second device. [Technology 15] A computer program for causing a first device controlling a second device to execute the following: obtain device information, which is information transmitted from the second device that has detected that a predetermined timing has been reached and indicates the functions of the second device; obtain instruction information regarding the control details of the second device; generate an input sentence including the device information of the second device and the instruction information; input the input sentence to a generation AI; and obtain output information from the generation AI indicating the control details for the second device. This computer program can utilize a generation AI to obtain output information indicating the control details for the second device, and can improve the ease with which the first device (in other words, input sentences to the generation AI) can follow changes in the second device.

[0089] The technology of the present disclosure can be applied to an information processing device or an information processing system.

[0090] 10 Equipment control system, 12 Equipment control device, 18 Sensor, 20 Controlled device, 50 Equipment information acquisition unit, 52 Spatial information acquisition unit, 54 Instruction information acquisition unit, 56 Prompt generation unit, 58 Answer acquisition unit, 60 Equipment control unit, 100 Sensor information, 101 Equipment information, 102 Spatial information, 104 Instruction information.

Claims

1. An information processing method in which a first device controlling a second device acquires device information, which is information transmitted from the second device that has detected that a predetermined first timing has been reached and which indicates the functions of the second device, acquires instruction information relating to the control content of the second device, generates an input sentence including the device information of the second device and the instruction information, inputs the input sentence to a generation AI, and acquires output information from the generation AI that indicates the control content for the second device.

2. The information processing method according to claim 1, wherein the first device controls the second device based on the output information.

3. An information processing method as described in claim 1 or 2, wherein the first device acquires sensor information, which is information regarding the surrounding situation of the second device, transmitted from a sensor device that detects that a predetermined second timing has been reached, and the first device generates an input sentence further including the sensor information.

4. The information processing method of claim 3, wherein the second timing includes at least one of the following: the time when the power of the sensor device is turned on; the time when the sensor device is connected to a communication network; the time when the sensor information shows a predetermined change; and the time when a predetermined time has elapsed since the previous second timing.

5. An information processing method according to claim 1 or 2, wherein the first device further acquires spatial information relating to a space in which the second device operates, and the first device generates an input sentence further including the spatial information.

6. An information processing method according to claim 1 or 2, wherein the first device acquires specified deletion information transmitted from the second device that has detected that the first device is transitioning to an uncontrollable state, and when the first device acquires the deletion information transmitted from the second device, the first device excludes the device information of the second device from the input sentence.

7. An information processing method in which a second device detects whether a predetermined first timing has been reached, and if it detects that the first timing has been reached, transmits device information indicating the functions of the second device to a first device, and obtains output information from the first device indicating the control content for the second device, the output information being generated by the first device inputting an input sentence including the device information and instruction information regarding the control content of the second device into a generation AI.

8. The information processing method according to any one of claims 1, 2 and 7, wherein the device information includes information relating to a location where the second device is installed.

9. The information processing method according to any one of claims 1, 2 and 7, wherein the device information includes information relating to control that can be executed by the second device.

10. The information processing method according to any one of claims 1, 2 and 7, wherein the device information includes information relating to a current control state of the second device.

11. An information processing method according to any one of claims 1, 2 and 7, wherein the first timing includes at least one of the following: the time when the second device is powered on; the time when the second device is connected to a communication network; the time when the second device is connected to the first device; the time when the state of the second device changes; and the time when a predetermined time has elapsed since the previous first timing.

12. An information processing method in which a second device detects whether a predetermined first timing has been reached, and if it detects that the first timing has been reached, transmits information regarding the control being executed by the second device to a first device, obtains instruction information regarding the control content of the second device from the first device, generates an input sentence including device information indicating the functions possessed by the second device and the instruction information, inputs the input sentence to a generation AI, and obtains output information indicating the control content for the second device from the generation AI.

13. The information processing method according to claim 12, further comprising obtaining information regarding the control of a third device different from the second device from the first device, and generating an input sentence including the device information of the second device, the instruction information, and the information regarding the control of the third device.

14. An information processing device comprising: a device information acquisition unit that acquires device information, which is information transmitted from a second device that has detected that a predetermined timing has been reached and which indicates the functions possessed by the second device; an instruction information acquisition unit that acquires instruction information regarding the control content of the second device; a generation unit that generates an input sentence including the device information of the second device and the instruction information; and an output information acquisition unit that inputs the input sentence to a generation AI and acquires output information from the generation AI that indicates the control content for the second device.

15. A computer program for causing a first device controlling a second device to perform the following operations: acquire device information, which is information transmitted from the second device that has detected that a predetermined timing has been reached and indicates the functions of the second device; acquire instruction information regarding the control content of the second device; generate an input sentence including the device information of the second device and the instruction information; input the input sentence to a generation AI; and acquire output information from the generation AI that indicates the control content for the second device.

Images