Air conditioner control system and information processing method

Abstract

To provide a control system or the like of an air conditioner capable of facilitating flexible control of the air conditioner.SOLUTION: The control system includes a control unit, an input interface for inputting data, and a storage unit for storing a prompt to be input to the large language model. The large language model is trained by machine learning to generate an output sentence in response to an input sentence. The control unit acquires, from data input via the input interface, a text that is based on the input data and relates to operation of the air conditioner, inputs the prompt and the acquired text to the large-scale language model to generate an output sentence including a control parameter used to control the air conditioner, and controls the operation of the air conditioner on the basis of the control parameter included in the output sentence.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present disclosure relates to an air conditioner control system and an information processing method. [Background technology]

[0002] Patent Document 1 discloses a control system that controls equipment such as air conditioners in a home using voice recognition. The control system of Patent Document 1 recognizes the voice of an unspecified speaker by recognizing voice input via a voice input microphone installed in the home as a pre-registered vocabulary. The control system assigns operation commands corresponding to the recognized vocabulary using a command conversion database and transmits them as operation signals to the equipment. Patent Document 1 aims to use the control system to eliminate the need for remote control operation and improve operability. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2001-285969 [Non-patent literature]

[0004] [Non-Patent Document 1] Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. 2023. Robust speech recognition via large-scale weak supervision. In Proceedings of the 40th International Conference on Machine Learning (ICML'23), Vol. 202. JMLR.org, Article 1182, 28492-28518 Summary of the Invention [Problem to be solved by the invention]

[0005] The present disclosure provides an air conditioner control system and information processing method that can facilitate flexible control of an air conditioner. [Means for solving the problem]

[0006] An air conditioner control system according to one aspect of the present disclosure includes a control unit, an input interface for inputting data, and a memory unit for storing prompts to be input to a large-scale language model. The large-scale language model is trained by machine learning to generate output sentences in response to input sentences. The control unit acquires, from the data input via the input interface, text based on the input data and related to the operation of the air conditioner. The control unit inputs the prompt and the acquired text to the large-scale language model, causing it to generate output sentences including control parameters used to control the air conditioner. The control unit controls the operation of the air conditioner based on the control parameters included in the output sentences.

[0007] An information processing method according to one aspect of the present disclosure is an information processing method for controlling an air conditioner, and is executed by a control unit of one or more computers. A memory unit of the one or more computers stores a prompt to be input to a large-scale language model. The large-scale language model is trained by machine learning to generate an output sentence in response to an input sentence. The information processing method includes: acquiring, by the control unit, text related to operation of the air conditioner from data input via an input interface of the one or more computers, the text being based on the input data; inputting the prompt and the acquired text into the large-scale language model to output an output sentence including a control parameter used to control the air conditioner; and controlling the operation of the air conditioner based on the control parameter included in the output sentence. [Effects of the Invention]

[0008] According to the air conditioner control system and information processing method disclosed herein, it is possible to facilitate flexible control of the air conditioner. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a block diagram illustrating a configuration of a control system according to a first embodiment of the present disclosure. [Figure 2] Block diagram showing the server configuration in the control system [Figure 3] A block diagram showing the configuration of a terminal device in a control system. [Figure 4] 1 is a flowchart illustrating an example of the operation of the control system according to the first embodiment; [Figure 5A] A diagram to explain an example of individual prompt settings for each user [Figure 5B] A diagram to explain an example of common prompt settings for all users [Figure 6] A diagram illustrating prompts input to a large-scale language model in a control system. [Figure 7] Diagram to explain the output results of a large-scale language model [Figure 8] 10 is a flowchart illustrating the operation of a control system according to a first modification of the first embodiment; [Figure 9] 10 is a flowchart illustrating the operation of a control system according to a second modification of the first embodiment; [Figure 10] 10 is a flowchart illustrating an example of an initial setting process in a control system according to a second embodiment. [Figure 11] 10 is a flowchart illustrating a registration process of individual settings in a control system according to a second embodiment. [Figure 12] 10 is a flowchart illustrating a setting change process in a control system according to a second embodiment. [Figure 13] 10 is a flowchart illustrating a setting change process in a control system according to a modification of the second embodiment. [Figure 14] 10 is a flowchart illustrating the operation of the control system according to the third embodiment. [Figure 15]10 is a flowchart illustrating the operation of a control system according to a modification of the third embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings.

[0011] (Embodiment 1) 1. Configuration FIG. 1 is a block diagram showing the configuration of an air conditioner control system 1 according to a first embodiment of the present disclosure. The control system 1 of this embodiment includes an air conditioner 10, a server 8, and a terminal device 7. In the control system 1, the air conditioner 10, the server 8, and the terminal device 7 are connected to each other so that data communication is possible. The control system 1 of this embodiment can be applied to control of the air conditioner 10 using voice spoken by a user. In this system 1, for example, the user's voice is input via the terminal device 7, and the air conditioner 10 is controlled in accordance with a control command for the air conditioner 10 generated by the server 8 in response to the voice.

[0012] 1-1. Air conditioner configuration The air conditioner 10 illustrated in Fig. 1 includes a memory unit 11, a control unit 12, a communication unit 13, and an indoor temperature sensor 14. The air conditioner 10 may further include various sensors, such as an outdoor temperature sensor, to perform various functions. The air conditioner 10 may also include a display for displaying visual information to a user.

[0013] The air conditioner 10 includes an indoor unit 20 and an outdoor unit 30. The memory unit 11, the control unit 12, the communication unit 13, and the indoor temperature sensor 14 may be provided in the indoor unit 20. The air conditioner 10 further includes a ventilation device 50, which is capable of supply ventilation that supplies outdoor air into the room and / or exhaust ventilation that exhausts indoor air to the outside.

[0014] The indoor unit 20 is provided with an indoor heat exchanger 22 that exchanges heat with indoor air, and a fan 24 that draws the indoor air into the indoor unit 20 and blows out the indoor air into the room after heat exchange with the indoor heat exchanger 22.

[0015] The outdoor unit 30 is provided with an outdoor heat exchanger 32 that exchanges heat with outdoor air, and a fan 34 that draws outdoor air into the outdoor unit 30 and blows the outdoor air out of the room after exchanging heat with the outdoor heat exchanger 32. The outdoor unit 30 is also provided with a compressor 36, an expansion valve 38, and a four-way valve 40 that execute a refrigeration cycle with the indoor heat exchanger 22 and the outdoor heat exchanger 32.

[0016] The indoor heat exchanger 22, the outdoor heat exchanger 32, the compressor 36, the expansion valve 38, and the four-way valve 40 are each connected by a refrigerant pipe through which a refrigerant flows. In cooling operation and dehumidification operation (weak cooling operation), the air conditioner 10 executes a refrigeration cycle in which the refrigerant flows from the compressor 36 through the four-way valve 40, the outdoor heat exchanger 32, the expansion valve 38, and the indoor heat exchanger 22 in that order, before returning to the compressor 36. In heating operation, the air conditioner 10 executes a refrigeration cycle in which the refrigerant flows from the compressor 36 through the four-way valve 40, the indoor heat exchanger 22, the expansion valve 38, and the outdoor heat exchanger 32 in that order, before returning to the compressor 36.

[0017] In this embodiment, the ventilation device 50 is installed outdoors together with the outdoor unit 30. In addition to its ventilation function, the ventilation device 50 can dehumidify or humidify the indoor air in the controlled space, for example, by supplying dehumidified outdoor air or moistened outdoor air to the controlled space.

[0018] The air conditioner 10 has, for example, a cooling function, a heating function, a dehumidifying function, an air cleaning function, and / or a ventilation function using the ventilation device 50. The air conditioner 10 may also have a dehumidifying function and / or a humidifying function using the ventilation device 50. The air conditioning operation performed by the air conditioner 10 can be selected by the user. These functions and the operating modes that perform various functions can be freely combined (for example, cooling and dehumidifying function, heating and humidifying function, cooling and ventilation mode, etc.). For example, in cooling mode, the air conditioner 10 can perform weak cooling operation (also known as compressor dehumidification or refrigeration cycle dehumidification) and dehumidification operation that throttles the expansion valve in addition to cooling operation.

[0019] The storage unit 11 is a recording medium that records various information and control programs, and may be a memory that functions as a work area for the control unit 12. The storage unit 11 is realized by, for example, a flash memory, a RAM (Random Access Memory), a ROM (Read Only Memory), other storage devices, or an appropriate combination of these.

[0020] The memory unit 11 may store standards or thresholds used for various controls of the air conditioner 10. The memory unit 11 may store information acquired from various sensors, such as the indoor temperature sensor 14. Information acquired from the terminal device 7, the server 8, or an external information source outside the control system 1 may be stored in the memory unit 11. The information acquired in this manner may be read out by the control unit 12 when controlling the air conditioner 10.

[0021] The storage unit 11 may store a computer program (sometimes abbreviated as a program in this disclosure) for causing the air conditioner 10 to execute the control method. The storage unit 11 may also include a non-transitory computer-readable storage medium in which the computer program is stored.

[0022] The control unit 12 is a controller that controls at least some of the functions of the air conditioner 10. The control unit 12 includes a general-purpose processor such as a CPU, MPU, MCU, FPGA, DSP, or ASIC that executes programs to achieve predetermined functions. The control unit 12 can implement various controls in the air conditioner 10 by calling and executing control programs stored in the storage unit 11. The control unit 12 also works in cooperation with the storage unit 11 to read and write data stored in the storage unit 11. The control unit 12 is not limited to a processor or the like that implements predetermined functions through cooperation between hardware and software, but may also be a hardware circuit designed specifically to implement predetermined functions.

[0023] The control unit 12 receives various control commands (for example, commands to activate various operation modes of the air conditioner 10 and / or temperature setting commands related to heating and cooling control) from the server 8 or the like via the communication unit 13. The control unit 12 controls each component of the air conditioner 10 so that the air conditioner 10 performs its cooling function, heating function, ventilation function, etc., based on set values ​​according to these control commands and detected values ​​received from various sensors (for example, indoor humidity and / or outdoor humidity). The set values ​​indicate, for example, target values ​​for controlling the operation mode and / or temperature of the air conditioner 10. The set values ​​are stored in the memory unit 11 and are updated according to control commands.

[0024] The communication unit 13 communicates with the air conditioner 10 and external devices such as the terminal device 7 and server 8, or external information sources, in accordance with a predetermined standard, transmitting and receiving data. The predetermined standard includes, but is not limited to, Wi-Fi (registered trademark), IEEE802.2, IEEE802.3, 3G, 4G, LTE, intranet, extranet, LAN, ISDN, VAN, CATV communication network, virtual private network, telephone line network, mobile communication network, satellite communication network, infrared and / or Bluetooth (registered trademark), etc. For example, the control unit 12 can cooperate with the server 8 and / or the terminal device 7 via the communication unit 13.

[0025] The indoor temperature sensor 14 detects the temperature of the indoor air drawn into the indoor unit 20 from the controlled space. In one embodiment, the indoor temperature sensor 14 is provided at the intake port of the indoor unit 20 through which the indoor air is drawn. Information detected by the indoor temperature sensor 14 is stored in the memory unit 11 and later used by the control unit 12 or transmitted to the terminal device 7 or the server 8.

[0026] In addition to the indoor temperature sensor 14, the air conditioner 10 may be equipped with sensors for acquiring various information from outside the air conditioner 10 in order to perform its functions. For example, the air conditioner 10 may include an outdoor air temperature sensor that detects the outdoor air temperature of the controlled space. These sensors, including the indoor temperature sensor 14, can acquire various information for operating the air conditioner 10.

[0027] 1-2. Server configuration 2 is a block diagram showing the configuration of the server 8 in the control system 1. The server 8 is configured with an information processing device such as a computer. The server 8 shown in FIG. 2 includes a control unit 80, a storage unit 81, and a communication unit 82.

[0028] The control unit 80 includes, for example, a CPU or MPU that works in cooperation with software to realize predetermined functions. The control unit 80 controls, for example, the overall operation of the server 8 to execute the information processing method in the control system 1 of this embodiment. The control unit 80 reads data and programs stored in the storage unit 81 and performs various arithmetic processing to realize various functions.

[0029] The control unit 80 executes a control program 88 including, for example, a set of instructions for implementing each of the above functions. The control program 88 may be provided via a communication network such as the Internet and stored in the storage unit 81, or may be stored in a portable recording medium. The control unit 80 may be a hardware circuit such as a dedicated electronic circuit or a reconfigurable electronic circuit designed to implement each of the above functions. The control unit 80 may be configured with various semiconductor integrated circuits such as a CPU, MPU, GPU, GPGPU, TPU, microcomputer, DSP, FPGA, and ASIC.

[0030] The storage unit 81 is a storage medium that stores programs and data necessary to realize the functions of the server 8. The storage unit 81 stores parameters, data, and control programs for realizing predetermined functions. The storage unit 81 includes, for example, an HDD or SSD. The storage unit 81 illustrated in FIG. 2 stores a large-scale language model (LLM) 86, a prompt database (DB) 87, and the above-mentioned control program 88.

[0031] The LLM 86 is a pre-trained language model that generates output sentences from input sentences through machine learning using a relatively large amount of training data. The LLM 86 is realized using, for example, a sequence transformation model including a self-attention mechanism, and may include various parameters and programs. The prompt DB 87 stores prompts input to the LLM 86 in association with users of the control system 1.

[0032] The storage unit 81 includes a RAM such as a DRAM or an SRAM, and temporarily stores (i.e., holds) data. The storage unit 81 may function as a work area for the control unit 80, and may include a storage area in the internal memory of the control unit 80.

[0033] The communication unit 82 is a module (circuit) that connects to an external device in accordance with a predetermined communication standard for wired or wireless communication. The predetermined communication standard includes, for example, USB, HDMI (registered trademark), IEEE802.11, Wi-Fi, Bluetooth, etc. The communication unit 82 may connect the server 8 to a communication network such as the Internet. The communication unit 82 is an example of an acquisition unit that receives various information from an external device or a communication network. The communication unit 82 may constitute an input interface for inputting various information in the control system 1 and / or an output interface for outputting various information.

[0034] In the server 8, the acquisition of various pieces of information may be realized not only by the communication unit 82 but also by cooperation with various software in the control unit 80, etc. The control unit 80 may acquire various pieces of information by reading out the various pieces of information stored in various storage media (for example, the storage unit 81) into the working area of ​​the control unit 80.

[0035] 1-3.Configuration of terminal device Fig. 3 is a block diagram showing the configuration of a terminal device 7 in the control system 1. The terminal device 7 is formed of an electronic device such as a smartphone, a tablet terminal, or a PC (personal computer). The terminal device 7 shown in Fig. 3 includes a control unit 70, a storage unit 71, a communication unit 72, an operation unit 73, a display unit 74, a microphone 75, and a speaker 76.

[0036] The control unit 70 includes, for example, a CPU or MPU that works in cooperation with software to realize predetermined functions. The control unit 70 controls, for example, the overall operation of the terminal device 7. The control unit 70 reads data and programs stored in the storage unit 71, performs various arithmetic processing, and realizes various functions.

[0037] The control unit 70 executes, for example, a program including a set of instructions for implementing each of the above functions. The program may be provided via a communication network such as the Internet, or may be stored on a portable recording medium. The control unit 70 may also be a hardware circuit, such as a dedicated electronic circuit or a reconfigurable electronic circuit, designed to implement each of the above functions. The control unit 70 may also be configured with various semiconductor integrated circuits, such as a CPU, MPU, GPU, GPGPU, TPU, microcomputer, DSP, FPGA, and ASIC.

[0038] The storage unit 71 is a storage medium that stores programs and data necessary to realize the functions of the terminal device 7. The storage unit 71 includes, for example, an HDD or SSD, and stores the above-mentioned programs and various data. The storage unit 71 also includes, for example, a RAM such as a DRAM or an SRAM, and temporarily stores (i.e., holds) data. The storage unit 71 may function as a work area for the control unit 70, and may include a storage area in the internal memory of the control unit 70.

[0039] The communication unit 72 is a module (circuit) that connects to an external device in accordance with a predetermined communication standard for wired or wireless communication. The predetermined communication standard includes, for example, USB, HDMI, IEEE802.11, Wi-Fi, Bluetooth, etc. The communication unit 72 may connect the terminal device 7 to a communication network such as the Internet.

[0040] The operation unit 73 is a general term for operation members operated by the user. The operation unit 73 includes, for example, a touch panel that is superimposed on the display unit 74 and that inputs various touch operations. The operation unit 73 may be a physical button or switch provided on the terminal device 7, or may include a connection unit that communicates with an external input device and receives operation signals. A keyboard, a mouse, a touchpad, or the like may be used as an input device. The operation unit 73 may be various GUIs such as virtual buttons, icons, cursors, software keyboards, and objects displayed on the display unit 74. The operation unit 73 may constitute an input interface of the control system 1.

[0041] The display unit 74 is configured with, for example, a liquid crystal display or an organic EL display. The display unit 74 may display various types of information, such as various GUIs for operating the operation unit 73 and information input from the operation unit 73. When using a display device external to the terminal device 7, for example, the display unit 74 may be an output interface circuit for a video signal or the like that complies with the HDMI standard or the like.

[0042] The microphone 75 includes, for example, one or more microphone elements built into the terminal device 7. The microphone 75 generates audio data based on an audio signal indicating the picked-up audio, and outputs the audio data to the control unit 70. The microphone 75 is an example of an input interface that inputs audio data. Instead of or in addition to the built-in microphone 75, the terminal device 7 may be provided with a connection unit such as a terminal for connecting to an external microphone.

[0043] The speaker 76 includes, for example, one or more speaker elements built into the digital camera 100, and outputs sound to the outside of the terminal device 7 under control of the control unit 70. The terminal device 7 may be provided with a connection unit for connecting to an external speaker, earphones, or the like instead of or in addition to the built-in speaker 76. In the control system 1, the speaker 76 is an example of an output interface that presents information to the user. In the control system 1, the microphone and / or speaker are not limited to being provided in the terminal device 7, and may also be provided in, for example, the air conditioner 10.

[0044] The above-described configurations of the air conditioner 10, server 8, and terminal device 7 are merely examples, and these configurations are not limited to the above examples. For example, each control unit 12, 80, 70 may include two or more processors. Furthermore, each storage unit 11, 81, 71 may store data and programs different from those in the above examples. The information processing method of this embodiment may be executed by any one of the control units 12, 80, 70, or may be executed by cooperative operation. Furthermore, the information processing method of this embodiment may be executed in distributed computing.

[0045] 2.Operation The control system 1 of this embodiment configured as above controls the air conditioner 10 using, for example, the LLM 86 stored in the server 8. The operation of the control system 1 will be described with reference to FIGS.

[0046] 4 is a flowchart illustrating the operation of the control system 1 of this embodiment. For example, when voice data is input via the microphone 75 in response to a user's speech, the control unit 70 of the terminal device 7 transmits the voice data to the server 8 via the communication unit 72. The processing of this flowchart is started, for example, when the voice data is received by the server 8, and each process is executed by the control unit 80.

[0047] The control unit 80 acquires text data generated by speech recognition processing as speech data based on speech data received via the communication unit 82 (S1). The speech recognition processing generates text data indicating the content of the utterance from the speech data. The speech recognition processing is performed, for example, by a speech recognition model that generates text data based on speech data. In step S1, the control unit 80 identifies a user by, for example, an identifier used for data communication with the terminal device 7, and acquires speech data associated with the user.

[0048] The above-mentioned speech recognition model can be realized by, for example, various neural networks, statistical models such as hidden Markov models and n-gram language models, or any combination thereof. For example, Whisper, disclosed in Non-Patent Document 1, may be used as a speech recognition model trained by machine learning. The speech recognition model may be stored in the storage unit 81 of the server 8, or may be stored in an external information processing device. In step S1, the control unit 80 may transmit speech data via the communication unit 82 to an information processing device having the speech recognition model, and acquire the generated speech data from the information processing device.

[0049] The control unit 80 determines whether the text of the utterance indicated by the utterance data includes a wake word that starts voice-based control of the air conditioner 10 (S2). For example, the wake word is a predetermined word such as "air conditioner," and is stored in the storage unit 81 by the initial setting processing of the air conditioner 10, which will be described later. If the utterance does not include a wake word (NO in S2), the control unit 80, for example, ends the processing of this flowchart, and executes the processing from step S1 onwards again the next time voice data is received.

[0050] If the utterance includes a wake word (YES in S2), the control unit 80 obtains a prompt associated with the user who made the utterance, for example, from the prompt DB 87 in the storage unit 81 (S3). In step S3, the control unit 80 determines the user associated with the utterance data of the utterance as the user who made the utterance. In the control system 1 of this embodiment, the prompt is set and stored in the prompt DB 87 before the processing of this flowchart is executed. Examples of setting such a prompt are shown in FIGS. 5A and 5B.

[0051] FIG. 5A is a diagram illustrating an example of prompt settings for individual users. The individual setting table T1 illustrated in FIG. 5A manages various setting information for prompts for each user. The individual setting table T1 stores, for each "personal ID" that identifies a user, the setting value of the air conditioner 10 according to, for example, a phrase included in an utterance, or the amount by which the setting value is changed. "DEF" in FIG. 5A indicates a default setting, which is set, for example, when the air conditioner 10 is manufactured or before shipping. Furthermore, "-" in the figure indicates an unset state.

[0052] Fig. 5B is a diagram illustrating an example of prompt settings common to all users. The common table T2 illustrated in Fig. 5B manages setting information and information indicating the operating status of the air conditioner 10 that are common to all users. This information common to all users can be used, for example, to notify users when an error occurs.

[0053] The control unit 80 executes a process to have the LLM 86 generate an output sentence based on the acquired prompt and utterance data (S4). The output sentence of the LLM 86 is generated so as to include various control parameters such as setting values ​​for controlling the air conditioner 10. In such LLM processing, the control unit 80 inputs the utterance data, for example, into the prompt as an input sentence to the LLM 86.

[0054] FIG. 6 is a diagram illustrating a prompt input to the LLM 86 in the control system 1. FIG. 6 illustrates a prompt D1 stored as text data in the prompt DB 87. The prompt D1 of this embodiment includes a "command statement" indicating an instruction for the LLM 86 to generate an output statement, and a "constraint" that conditions the generation of the output statement. In the example of FIG. 6, the command statement is written to cause the LLM 86 to generate an output statement including control parameters for each of the items "ON-OFF," "air volume," "air direction," and "temperature" of the air conditioner 10. The constraints describe the output format and options for each control parameter. The prompt D1 of this example can further include utterance data 45.

[0055] When the control unit 80 acquires a prompt D1 associated with a user of utterance data (S3), the control unit 80 may generate a user-specific prompt D1 from a template or the like common to each user. For example, the control unit 80 may refer to an individual setting table T1 illustrated in FIG. 5A and change the constraints from the template according to the user based on the setting values ​​corresponding to the user. Not limited to the above example, the prompt D1 may be stored in advance in the prompt DB 87 for each user. Furthermore, information corresponding to the individual setting table T1 may be stored, for example, as a database query or an embedded representation similar to the multidimensional feature representation into which an input sentence is converted in the LLM 86.

[0056] FIG. 7 is a diagram illustrating the output result by the LLM 86. FIG. 7 illustrates an example of an output sentence D2 output as an output result from the LLM 86 when a prompt D1 including utterance data 45 such as that shown in FIG. 6 is input. The output sentence D2 includes a "control result" that indicates the control parameters 25 of the air conditioner 10 generated by the LLM 86 for each control item. The output sentence D2 in this example is generated so as to further include an explanation of why the control parameters 25 in the "control result" were determined from the utterance data 45. Furthermore, in this example, the LLM 86 outputs a psychological evaluation, such as a positive or negative emotion, of the user who made the utterance regarding the operation of the air conditioner 10 based on the utterance data 45.

[0057] Returning to Fig. 4, the control unit 80 generates a control command to the air conditioner 10 from the output result of the LLM 86 (S5). For example, the control unit 80 generates a control command for each control item based on the control parameters 25 in the output statement D2.

[0058] The control unit 80 transmits the generated control command to the air conditioner 10 via the communication unit 82 (S6). Before transmitting the control command, confirmation may be made, for example, through interaction with the user via the terminal device 7 using various voice operations or GUI displays, to confirm whether or not the user agrees to changes to the settings of the air conditioner 10, etc., made by the control command.

[0059] When the control unit 80 transmits the control command (S6), the process of this flowchart ends. The server 8 may repeatedly execute the processes from step S1 onward, for example, in response to the reception of voice data. Furthermore, if the control parameters 25 of all control items specified by the prompt D1 are not output in the output statement D2, the user may be prompted to speak by voice output via the terminal device 7, for example, to reacquire speech data, and the LLM process (S4) may be executed again. The "constraint" of the prompt D1 may be described so as to cause the LLM 86 to execute such a process.

[0060] According to the above processing, a prompt D1 for each user is acquired (S3) in response to the user's utterance (S1, S2), and a control command is generated (S4, S5) from the output result of the LLM 86 based on the prompt D1 and the utterance data, and transmitted to the air conditioner 10 (S6). This makes it possible to control the air conditioner 10 in accordance with the user's preferences, for example, from the prompt D1 set for each user in the control system 1. Furthermore, by generating control parameters 25 in the control command using the LLM 86 (S4, S5), the air conditioner 10 can be operated using natural language without using specific operation commands or keywords, thereby improving user convenience.

[0061] In the above example, the control unit 80 inputs the text of the utterance data into the prompt D1 illustrated in Fig. 5 to the LLM 86 (S4). The utterance data may be input to the LLM 86 separately from the prompt D1. For example, the utterance data may be input by specifying in the prompt D1 the storage location of a text file indicating the utterance data.

[0062] In the above example, if the utterance includes a wake word (YES in S2), the processes from step S3 onward are executed. For example, if the utterance data is acquired while a predetermined user operation is being input, such as while a virtual button on a GUI display on the terminal device 7 is being pressed (S1), the processes from step S3 onward may be executed without making the determination in step S2.

[0063] 2-1. Variation 1 In the above-described first embodiment, an example has been described in which the LLM process is executed for each utterance of the user (S4). The LLM process may be performed based on a dialogue history including past utterances and responses by the LLM 86. A first modification of the first embodiment will be described with reference to FIG. 8.

[0064] 8 is a flowchart illustrating the operation of the control system of Modification 1 of Embodiment 1. In this modification, in addition to the same processes (S1 to S3, S5 to S6) as those in Embodiment 1 (FIG. 4), a process of accumulating a dialogue history (S51) is executed. Also, instead of the LLM process of inputting prompt D1 and utterance data (S4), an LLM process of inputting a dialogue history is further executed (S4A).

[0065] For example, when the control unit 80 acquires a prompt D1 associated with the user who made the utterance (S3), it inputs the acquired prompt D1 and the utterance data of the utterance as well as a past dialogue history with the user to the LLM 86 to generate an output sentence D2 (S4A). The control unit 80 may acquire the dialogue history of the user by, for example, referring to the storage unit 81, and input the dialogue history to the LLM 86 together with the prompt D1.

[0066] The control unit 80 associates the utterance data acquired in step S1 with the output result of the LLM 86 in step S4A, and stores the associated utterance data as a dialogue history of the user corresponding to the utterance data in the storage unit 81 (S51). The control unit 80 adds the utterance data and the output result to the dialogue history stored in the storage unit 81, for example, every time step S51 is executed.

[0067] According to the above processing, an output result is obtained by the LLM 86 based on an input sentence including the past dialogue history between the user and the LLM 86 (S4A), and it is further possible to make it easier to control the air conditioner 10 flexibly according to the user's preferences.

[0068] In the above process, the dialogue history may be stored using the memory functions of various frameworks or libraries such as LangChain and LlamaIndex in application development using LLM.

[0069] 2-2. Variation 2 In the above-described first embodiment, the LLM process (S4) for inputting the prompt D1 and speech data for each user has been described. The information input to the LLM 86 is not limited to this, and may further include information acquired from an information source external to the LLM 86. For example, a technique such as RAG (Retrieval Augmented Generation) may be used to generate the output result by the LLM 86. A second modification of the first embodiment will now be described with reference to FIG. 9.

[0070] 9 is a flowchart illustrating the operation of the control system of Modification 2 of Embodiment 1. In this modification, in addition to the same processes (S1 to S3, S5 to S6) as in FIG. 4, information related to the utterance is acquired from an external information source (S61), and instead of the LLM process of step S4, an LLM process is executed in which the acquired information is further input (S4B).

[0071] For example, when the control unit 80 acquires a user's utterance data and a prompt D1 (S1, S3), it acquires additional information related to the utterance indicated by the utterance data from an external information source (S61). The external information source is, for example, a database constructed in the storage unit 81 based on information input to the server 8 via the communication unit 82 before executing the processing of this flowchart. In step S61, the control unit 80 acquires the additional information using various database search techniques. The database may be constructed as a vector database, or it may not be constructed in advance. For example, in step S61, the control unit 80 may acquire information on the World Wide Web (WWW) via the communication unit 82.

[0072] The control unit 80 inputs the acquired additional information to the LLM 86 in addition to the prompt D1 and the utterance data, and causes the LLM 86 to generate an output sentence D2 (S4B). The control unit 80 may input the additional information by including it in the prompt D1.

[0073] According to the above process, additional information related to the utterance is obtained from an external information source (S61) and input to the LLM 86 in addition to the utterance data and prompt D1 (S4B). As a result, even information not included in the training data of the LLM 86 can be input as additional information, thereby enabling the generation of output sentences D2 corresponding to the utterance data with greater accuracy. For example, additional information obtained by accessing the latest information in an external information source can keep the response output from the LLM 86 in response to the user utterance indicated by the utterance data up to date and highly relevant to the utterance.

[0074] In the above example, a prompt is acquired (S3), and then additional information is acquired (S61). The acquisition of additional information is not limited to the above example, and it is sufficient that utterance data is acquired (S1). For example, the process of step S61 may be executed when the utterance includes a wake word (YES in S2).

[0075] The above describes an example in which external information other than the prompt D1 and utterance data is used by retrieval expansion and generation (RAG). The introduction of external information may be realized by fine-tuning LLM86 or the like based on additional training data intended for the control of the air conditioner 10, for example, or may be achieved by combining RAG and fine-tuning or the like.

[0076] 3. Summary As described above, in this embodiment, the control system 1 for the air conditioner 10 includes a control unit 80, a microphone 75 as an example of an input interface for inputting voice data as an example of data, and a memory unit 81 for storing prompts to be input to an LLM 86 as an example of a large-scale language model. The LLM 86 is trained by machine learning to generate output sentences based on input sentences. The control unit 80 acquires speech data, which is an example of text based on the input voice data and related to the operation of the air conditioner 10, from the voice data input via the microphone 75 (S1). The control unit 80 inputs the prompt and the acquired speech data to the LLM 86, causing it to generate an output sentence including control parameters used to control the air conditioner 10 (S4, S4A, S4B). The control unit 80 controls the operation of the air conditioner 10 based on the control parameters included in the output sentence (S6, S7).

[0077] According to the above-described control system 1, even if the speech data relating to the operation of the air conditioner 10 contains ambiguous expressions or instructions, the LLM 86 can convert them into control parameters, making it easier to flexibly control the air conditioner 10.

[0078] In this embodiment, the storage unit 81 stores prompts that are input to the LLM 86 in association with each of a plurality of users (see FIG. 5A). The control unit 80 acquires utterance data in association with each of a plurality of users (S1), and inputs, from the prompts stored in the storage unit 81, prompts that correspond to the users associated with the acquired utterance data to the LLM 86 (S3, S4, S4A, S4B). This allows the LLM 86 to generate output statements that include control parameters for the air conditioner 10, for example, in accordance with prompts set for each user.

[0079] In this embodiment, the prompt includes an example of a command statement that causes the LLM 86 to generate an output statement D2 that includes the control parameter 25, and a constraint that is an example of a constraint that conditions the generation of the output statement D2, as shown, for example, in prompt D1 in FIG. 6 and output statement D2 in FIG. 7.

[0080] In a first modification of this embodiment, the storage unit 81 stores a dialogue history, which is an example of a history associating acquired utterance data with output sentences generated by the LLM 86 from the utterance data (see S51). The control unit 80 further inputs the dialogue history stored in the storage unit 81 to the LLM 86, causing it to generate output sentences (S4A). This allows output sentences to be generated by referring to past dialogue history in response to, for example, a user's utterance, making it easier to perform interactive control in the control system 1.

[0081] In a second modification of this embodiment, the control unit 80 acquires additional information related to the utterance data based on the acquired utterance data (S61), and further inputs the additional information to the LLM 86 to generate an output sentence (S4B). As a result, even if the information is not included in the training data of the LLM 86, information related to the control of the air conditioner 10 can be used as additional information in accordance with the utterance data, making it possible to generate an output sentence with even greater accuracy.

[0082] In this embodiment, the microphone 75 inputs voice data of the utterance indicated by the speech data. This allows the user to operate the air conditioner 10 by voice, for example, without using a remote controller or the like for the air conditioner 10.

[0083] In this embodiment, the control system 1 includes a server 8, which is an example of an information processing device, a terminal device 7, and an air conditioner 10, and the server 8, terminal device 7, and air conditioner 10 are connected to each other so that data can be communicated. As shown in FIG. 2, for example, the server 8 includes a control unit 80 and a communication unit 82, which is an example of a communication interface that communicates data with external devices such as the terminal device 7 and the air conditioner 10. As shown in FIG. 3, for example, the terminal device 7 includes a microphone 75, which is an example of an input interface, and a communication unit 72, which is an example of a communication interface that communicates data with external devices such as the server 8 and the air conditioner 10. As shown in FIG. 1, for example, the air conditioner 10 includes a memory unit 11 that stores setting values ​​used to control the air conditioner 10, and a communication unit 13, which is an example of a communication interface that communicates data with external devices such as the server 8 and the terminal device 7.

[0084] The control system 1 of this embodiment provides an information processing method for controlling the air conditioner 10 (S1 to S6). This method is executed by a control unit 80 of a server 8, which is an example of one or more computers. A memory unit 81 of the server 8 stores prompts to be input to an LLM 86, which is an example of a large-scale language model, for example, in a prompt DB 87 shown in FIG. 2. The LLM 86 is trained by machine learning to generate output sentences in response to input sentences. This method includes the control unit 80 acquiring, from voice data, which is an example of data input by a user via a microphone 75, which is an example of an input interface of a terminal device 7, which is an example of one or more computers, speech data, which is an example of text based on the input voice data and is related to the operation of the air conditioner 10 (S1). This method also includes inputting the prompt and the acquired speech data into the LLM 86, and causing it to output an output sentence including a control parameter used to control the air conditioner 10 (S4, S4A, S4B). This method also includes controlling the operation of the air conditioner 10 based on the control parameter included in the output sentence (S5, S6).

[0085] In this embodiment, a control program 88 is provided as an example of a program for causing the control unit 80 to execute the information processing method described above. According to the information processing method and program described above, control parameters for the air conditioner 10 can be output by the LLM 86 from user speech data, making it easier for the user to flexibly control the air conditioner 10.

[0086] (Embodiment 2) In the above-described first embodiment, the operation of the control system 1 is described, in which a prompt set for each user is input to the LLM 86 to control the air conditioner 10. In the second embodiment, the operation of the control system 1 for setting a prompt for each user is described.

[0087] 1. Initial setting process The control system 1 of this embodiment first performs an initial setting process to set the initial state of a prompt in association with a user, and then performs a setting change process to change the setting in response to the user's utterance, etc. The system 1 sets the prompt based on the user's utterance using the LLM 86.

[0088] 10 is a flowchart illustrating an example of an initial setting process in the control system 1 of embodiment 2. The process of this flowchart is started, for example, when a communication connection with the air conditioner 10 is established in the server 8. Each process of this flowchart is executed, for example, by the control unit 80 while acquiring user utterance data as needed, similar to step S1 of FIG.

[0089] First, the control unit 80 sets a wake word for the air conditioner 10 and stores it in the storage unit 81 or the like (S11). The wake word may be set based on speech data acquired through interaction with the user by the LLM 86, for example, via the terminal device 7, or may be set by a user operation on the operation unit 73 by displaying a setting screen or the like on the display unit 74.

[0090] Next, the control unit 80 acquires various setting information for the prompts for each user based on, for example, the speech data for various controls and various operating modes of the air conditioner 10, and registers the setting information in the individual setting table T1 illustrated in FIG. 5A (S12). For example, if a user utters "Change the setting from 'immediately' to '0.5'" for the item "Speech: Immediately" in the individual setting table T1 of FIG. 5A, setting information associating the item with a setting value or a change amount for the setting value is acquired. Then, based on the setting information, the setting value for the item associated with the user's personal ID is changed from the default setting to "0.5." Details of this individual setting registration process (S12) will be described later.

[0091] At the start of step S12, based on the personal ID of the user whose speech data has been acquired, the current setting value associated with that personal ID in the individual setting table T1 may be notified to that user by voice data or the like.

[0092] The control unit 80 notifies the user of the completion of the registration process by, for example, transmitting audio data to the terminal device 7 via the speaker 76 or a message to be displayed on the display unit 74 via the communication unit 82 (S13). The notification to the user is not limited to after the individual setting registration process (S12) is executed, but may also be executed during the execution of the process (S12), in response to the user's utterance, when registering each item in the individual setting table T1. In the example of FIG. 5A described above, audio data of the notification such as "The setting of 'immediately' has been changed to '0.5'" may be output.

[0093] The control unit 80 registers, in the storage unit 81 (S14), an individual setting table T1 that corresponds to the prompt and is referenced in acquiring the prompt (S3), in association with the user of the utterance data. In step S14, the control unit 80 may generate a prompt associated with the user based on the individual setting table T1, and store the generated prompt in the prompt DB 87. Thereafter, the control unit 80 terminates the processing of this flowchart.

[0094] According to the above initial setting process, individual prompts can be set for each user in the control system 1.

[0095] The above initial setting process may be executed by, for example, writing a control prompt D1 for the air conditioner 10 as shown in Fig. 6 so that the LLM 86 will interact with the user in each process. Furthermore, a program for setting the control prompt D1 and information specific to the model of the air conditioner 10 may be stored in the storage unit 81. Also, a start flag or the like indicating the start of prompt setting may be output to the LLM 86 in response to a user's utterance, and the program may be executed by activating the start flag. For example, the start flag may be activated when a personal ID, items and setting values ​​in the individual setting table T1, etc. are acquired from the utterance data.

[0096] After registering setting information including items, setting values, etc. in the individual setting table T1, the program may activate a completion flag indicating completion of the setting. By activating the completion flag, the server 8 may cause the LLM 86 to notify the user that the setting is complete. Furthermore, the program is not limited to this. For example, in cases where a prompt is held according to the input format to the LLM 86, the LLM 86 may execute the above-described processing including the individual setting registration processing (S12).

[0097] 1-1. Individual setting registration process The individual setting registration process in step S12 of FIG. 10 will be described in detail with reference to FIG.

[0098] FIG. 11 is a flowchart illustrating the registration process (S12) of the individual settings in the control system 1 according to the second embodiment.

[0099] The control unit 80 acquires user utterance data through voice recognition processing based on voice data input via the microphone 75 of the terminal device 7 (S40), similar to, for example, step S1 in Fig. 4. For example, as described above with respect to the individual setting table T1 shown in Fig. 5A, user utterance data instructing operation methods for various controls and various operation modes of the air conditioner 10 is acquired.

[0100] The control unit 80 sets control items according to the content of the utterances made during control of the air conditioner 10, for example, by text analysis processing of the acquired utterance data (S41). Items set in this basic control setting (S41) are indicated by names including "utterance:" in the individual setting table T1 of Fig. 5A, for example. The text analysis processing may include syntax analysis of the utterance data and keyword extraction, or may be performed by inputting prompts and utterance data for the individual setting registration processing (S12) into the LLM 86, which then outputs control items, setting values, etc.

[0101] Furthermore, the control unit 80 sets control items for when, for example, the air conditioner 10 is controlled in accordance with the results of emotion analysis of utterance data during control (S42). In the output result of the LLM 86 shown in Fig. 7, the results of the emotion analysis are output as an evaluation of the user's psychology. Items set in this emotion control setting (S42) are indicated by names including "emotion:" in the individual setting table T1 of Fig. 5A, for example.

[0102] The control unit 80 also sets control items such as temperature and air volume when the air conditioner 10 is set to an operation mode such as away mode, which is intended for when the user is not present (S43). The control unit 80 also sets control items when the air conditioner 10 is set to an operation mode such as terminal-linked mode, which operates based on sensor information from various sensors mounted on the terminal device 7, etc. (S44). The control unit 80 then ends the processing of this flowchart and proceeds to step S13 in Fig. 10.

[0103] According to the individual setting registration process (S12) described above, various control items for the air conditioner 10 can be set and registered as setting values ​​or amounts of change thereto in the individual setting table T1 shown in FIG. 5A, for example.

[0104] In the individual setting registration process (S12), the order in which the processes (S41 to S44) are executed is not limited to the example described above, and each process can be executed appropriately based on the acquired speech data, for example. Before executing each process (S41 to S44), speech data may be acquired as needed, as in step S40. In this embodiment, emotion control setting (S42) may not be performed, and processes such as steps S43 and S44 may not be executed depending on the operating mode of the air conditioner 10. Furthermore, settings may be made for other control items in addition to or instead of the control items described for each of the above processes (S41 to S44). For example, dialects or unique control commands may be registered for each user.

[0105] Furthermore, the control unit 80 may cause, for example, the LLM 86 to output control items and corresponding setting values ​​based on the utterance data. Also, when a user utters an utterance to the effect that they want to share settings with other users, the settings may be registered in the individual setting table T1 in association with the multiple users.

[0106] For control items that are not included in the user's speech, the LLM 86 may generate text that prompts the user to speak, and the text data or voice data converted from the text data may be transmitted to the terminal device 7. Alternatively, for such control items, the default settings may be maintained.

[0107] 2. Setting change processing The setting change process executed after the above-described initial setting process (FIG. 10) will be described with reference to FIG.

[0108] 12 is a flowchart illustrating a setting change process in the control system 1 of the second embodiment. The process of this flowchart starts after, for example, the initial setting process (FIG. 10) has registered initial values ​​for each user in the individual setting table T1 illustrated in FIG. 5A, and a prompt for each user has been registered in the prompt DB 87 corresponding to the individual setting table T1. Each process of this flowchart is executed by the control unit 80 of the server 8.

[0109] The control unit 80 acquires user utterance data (S21), for example, in the same manner as in step S1 of FIG. 4, etc.

[0110] Based on the acquired speech data, the control unit 80 determines whether or not an instruction to change the settings of the control items of the air conditioner 10 has been issued (S22). Whether or not such an instruction to change the settings has been issued is determined, for example, by whether or not the speech data contains a keyword such as "change."

[0111] If a setting change is instructed (YES in S22), the control unit 80 acquires a prompt associated with the user corresponding to the utterance data, for example, in the same manner as in step S3 (S23).

[0112] For example, in the same individual setting registration process as in step S12, the control unit 80 changes the setting values ​​etc. set in the initial setting process instead of changing the default settings in step S12 (S24).

[0113] The control unit 80 notifies the user of the completion of the setting change, for example, in the same manner as in step S13 (25).

[0114] The control unit 80 updates the user's prompt corresponding to the utterance data in the prompt DB 87 based on, for example, the individual setting table T1 set by the individual setting registration process in step S24 (S26). The control unit 80 updates the prompt by changing or adding to the prompt based on, for example, the individual setting table T1 changed in step S24. For example, in the prompt D1 shown in FIG. 6, the control unit 80 changes the command statement and / or the constraint condition.

[0115] After updating the prompt in accordance with the setting change (S26), the control unit 80 ends the processing of this flowchart. If a setting change is not instructed (NO in S22), the control unit 80 also ends the processing of this flowchart. After that, for example, by acquiring speech data again (S21), the processing from step S21 onwards is executed.

[0116] According to the setting change process described above, the settings for various control items can be updated by the user's speech in prompts for each user.

[0117] 3. Variations In the above-described second embodiment, an example has been described in which the setting change process is performed in response to a user's utterance in the control system 1 (FIG. 12). The setting change process may be initiated based on a dialogue history including, for example, past utterances and responses by the LLM 86. A modification of this second embodiment will be described with reference to FIG. 13.

[0118] 13 is a flowchart illustrating a setting change process in the control system 1 of a modification of the second embodiment. The process of this flowchart starts in a state where the dialogue history has been stored in the storage unit 81 by, for example, a process (S51) for accumulating the dialogue history similar to that of the first modification of the first embodiment (FIG. 8). The setting change process of this modification includes processes (S21 to S26) similar to those of the second embodiment (FIG. 12) as well as processes related to the start of the setting change (S70, S71).

[0119] The control unit 80 determines whether or not the dialogue history stored for each user in the storage unit 81 satisfies a predetermined condition for changing the settings (S70). The predetermined condition is set in advance, for example, from the perspective of easily realizing settings for the air conditioner 10 that suit the user's preferences, such as a condition in which an utterance that generates the same control command is made a predetermined number of times (for example, five times) or more within a predetermined period such as one day. If the dialogue history does not satisfy the predetermined condition (NO in S70), the control unit 80 performs a setting change process in accordance with the user's utterance, for example, similar to the process shown in FIG. 12 (S21 to S26).

[0120] If the dialogue history satisfies a predetermined condition (YES in S70), the control unit 80 transmits a notification to the terminal device 7 or the like via, for example, the communication units 82 and 72, to guide the user to change the settings (S71). For example, in the terminal device 7, voice data indicating the voice is transmitted so that the speaker 76 outputs a voice such as "Do you want to change the setting value?". Thereafter, the control unit 80 acquires, for example, speech data of the user responding to the guidance (S21), and if a setting change is instructed (YES in S22), updates the prompt associated with the user (S23 to S25).

[0121] According to the above processing, in the control system 1, the setting change processing can be initiated based on the history of dialogue with the user by the LLM 86, making it easier to reflect the user's preferred settings for the air conditioner 10 in the prompts for that user.

[0122] In the above, an example has been described in which, when the dialogue history satisfies a predetermined condition (YES in S70), the user is notified of a setting change guide (S71) and an utterance from the user is acquired (S21). For example, the control unit 80 may not issue the notification in step S71, and may automatically update the prompt based on the dialogue history that satisfies the predetermined condition.

[0123] Furthermore, the process for starting the setting change (S70, S71) may be executed after acquiring user utterance data during control (S1), for example. In this case, the predetermined condition may be set such that the utterance during control does not match the output result from the LLM 86 immediately before in the dialogue history. For example, if an utterance such as "The temperature is too high" is made immediately after an output result to increase the temperature, it may be determined that the dialogue history satisfies the predetermined condition.

[0124] 4. Summary As described above, in this embodiment, the control system 1 for the air conditioner 10 includes a control unit 80, a storage unit 81, and a microphone 75, which is an example of an input interface that inputs voice data as an example of data. The air conditioner 10 is controlled based on output sentences from an LLM 86, which is an example of a large-scale language model trained by machine learning to generate output sentences in response to input sentences (S1 to S6). The control unit 80 acquires speech data, which is an example of text instructing how to operate the air conditioner 10, from the voice data input via the microphone 75 (S12, S40, S21), and registers prompts used in input sentences to the LLM 86 in the storage unit 81 (S12, S14) or updates them (S24, S26) in accordance with the acquired speech data.

[0125] According to the control system 1 as described above, for example, the prompt input to the LLM 86 can be set in accordance with the speech data, and the air conditioner 10 can be controlled based on the sentence output from the LLM 86 in response to the prompt.

[0126] In this embodiment, the storage unit 81 stores prompts that are input to the LLM 86 in association with each of a plurality of users. The control unit 80 acquires utterance data in association with each of the plurality of users (S40, S21), and registers or updates prompts corresponding to the users associated with the acquired utterance data (S12, S14, S23, S26). This allows prompts to be set for each user.

[0127] In a modification of this embodiment, the control system 1 further includes a speaker 76 as an example of an output interface for presenting information to the user. The control unit 80 notifies the user of a setting change guide (S71) as an example of outputting guidance information that guides the user to change the prompt to the speaker 76 via the communication unit 82 and the communication unit 72. This makes it easier for the user to change the prompt settings in accordance with the guidance information, for example.

[0128] In a modification of this embodiment, the storage unit 81 stores a dialogue history that is an example of a history that associates acquired utterance data with output sentences generated from the utterance data by the LLM 86. The control unit 80 outputs guidance information in accordance with the dialogue history stored in the storage unit 81 and the acquired utterance data (S70, S71). This makes it possible to prompt the user to change the prompt settings in accordance with the dialogue history and utterance data, for example, when a similar utterance and the generation of a corresponding output sentence from the LLM 86 are repeated.

[0129] In this embodiment, the prompt includes, for example, a command statement that is an example of a command for causing the LLM 86 to generate an output statement D2 that includes control parameters 25 used to control the air conditioner 10, and a constraint condition that is an example of a constraint that conditions the generation of the output statement D2, as shown in prompt D1 in Fig. 6 and output statement D2 in Fig. 7. The control unit 80 changes at least one of the command statement and the constraint condition of the prompt D1, for example, in a setting change process (S26).

[0130] The control system 1 of this embodiment provides an information processing method for controlling the air conditioner 10 (S11 to S14, S21 to S26, S70 to S71). This method is executed by the control unit 80 of the server 8 (an example of one or more computers). The air conditioner 10 is controlled based on output sentences from an LLM 86 that has been trained by machine learning to generate output sentences in response to input sentences (S1 to S6). This method includes the control unit 80 acquiring, from voice data that is an example of data input via a microphone 75, which is an example of an input interface of a terminal device 7 (an example of one or more computers), speech data that is an example of text based on data and that instructs how to operate the air conditioner 10 (S12, S40, S21). This method also includes the control unit 80 registering, in the storage unit 81 (S12, S14), or updating (S24, S26), a prompt to be used in the input sentence to the LLM 86 in accordance with the acquired speech data.

[0131] In this embodiment, a control program 88 is provided as an example of a program for causing the control unit 80 to execute the above-described information processing method. The above-described information processing method and program allow the user to set and change the prompts to be input to the LLM 86, making it easier for the user to flexibly control the air conditioner 10.

[0132] (Embodiment 3) In the above-described first embodiment, an example has been described in which the LLM process is executed based on a prompt and an utterance (S4) in the control system 1, and the content of the utterance is mainly reflected in the control of the air conditioner 10 based on the output result of the LLM 86 (see FIG. 6). In a third embodiment, an example will be described in which the control system 1 further reflects the user's emotions in the control, such as psychological evaluation based on utterance data.

[0133] 1.Operation 14 is a flowchart illustrating the operation of the control system 1 according to embodiment 3. In this embodiment, instead of the LLM processing and generation of control commands (S4, S5) in embodiment 1, for example, processing for reflecting the user's emotions in control (S4C, S5A) is further executed.

[0134] The control unit 80 performs LLM processing (S4C) based on, for example, the user's prompt and speech data so that the output sentence from the LLM 86 includes control parameters that reflect the user's emotions. For example, in a prompt similar to prompt D1 shown in Fig. 6, by setting a constraint that reflects the "psychological" evaluation in the control, such LLM processing (S4C) with emotion control can be realized.

[0135] The control unit 80 generates a control command for the air conditioner 10 from the output result of the LLM 86 that reflects the emotion obtained in step S4C (S5A). For example, in a control command to change a setting value of the air conditioner 10, if the output result includes both a change amount corresponding to the content of the utterance and a change amount corresponding to the emotion, the two are summed. For example, if the prompt corresponding to the individual setting table T1 in FIG. 5A outputs the LLM 86 for the user with personal ID "****0," with the utterance containing "a little" and the emotion being "positive," the change amount is calculated as "1.0." Such calculation rules may be written in the constraint conditions of the prompt, or the constraint conditions may output each change amount and then sum the two.

[0136] In step S5A of this embodiment, not limited to the above example, when the control differs between the content of the utterance and the emotion, the content of the utterance may be reflected in the control preferentially. Also, when the absolute value of the sum of the change amounts corresponding to each becomes a predetermined value or more (for example, the change amount for temperature is 5°C or more), the change amount may be limited to a predetermined maximum value (for example, 4°C).

[0137] According to the above processing, control parameters that reflect the results of emotion analysis in addition to the content of the utterance are obtained, for example, by the LLM 86, based on the user's utterance data, and control commands for the air conditioner 10 are generated based on the control parameters (S4C, S5A). This makes it even easier to control the air conditioner 10 in accordance with the user's intentions.

[0138] 2. Variations In the above, an example has been described in which the result of sentiment analysis is obtained by the LLM 86 based on speech data as text data (S4C). The sentiment analysis process may also be performed based on speech audio data. Such a modification of the third embodiment will be described with reference to FIG. 15.

[0139] 15 is a flowchart illustrating the operation of the control system 1 according to a modification of the third embodiment. In this modification, in addition to the same processes (S1 to S3, S5A to S6) as those in the third embodiment, emotion analysis processing from voice data (S81) is performed. Furthermore, in this modification, instead of the LLM processing (S4C) in the third embodiment that includes emotion analysis based on prompts and utterances, an LLM processing (S4D) based on emotion analysis results from prompts, utterances, and voice data is performed.

[0140] The control unit 80 acquires an analysis result such as positive or negative for the voice data from various feature quantities in the voice data, based on the voice data corresponding to the user's utterance data acquired in step S1, for example (S81). The voice data is input by the user through the microphone 75 of the terminal device 7 and transmitted to the server 8. The control unit 80 may transmit the voice data to an external application by data communication via the communication unit 82, and acquire the analysis result from the application.

[0141] The control unit 80 executes LLM processing based on, for example, the prompt, speech data, and the emotion analysis result acquired in step S81 (S4D). For example, a prompt such as that shown in Fig. 6 includes a constraint that reflects the emotion analysis result in the control of the air conditioner 10, and includes text indicating the emotion analysis result in addition to the user's speech content.

[0142] According to the above processing, emotion analysis processing can be performed with high accuracy based on voice data that can contain a variety of information related to emotions, and the user's emotions can be more easily reflected in the control of the air conditioner 10.

[0143] The above describes an example in which emotion analysis processing is performed based on voice data (S81). For example, speech data may be sent to an application external to the LLM 86, and the analysis results obtained by the application performing emotion analysis processing may be acquired. In this case, as in step S4D, LLM processing may be performed to input the emotion analysis results into the LLM 86. Furthermore, emotion analysis processing may be performed based on both voice data and speech data, rather than on either voice data or speech data. Furthermore, for example, various sensors that measure the user's heart rate may be installed in the terminal device 7, and the air conditioner 10 may be controlled by combining sensor information measured by the sensors with the emotion analysis results.

[0144] 3. Summary As described above, in this embodiment, the control system 1 for the air conditioner 10 includes a control unit 80, a microphone 75 as an example of an input interface for inputting voice data as an example of data, and a memory unit 81 for storing prompts to be input to an LLM 86 as an example of a large-scale language model. The LLM 86 is trained by machine learning to generate output sentences based on input sentences. From the voice data input via the microphone 75, speech data for an example of text related to the operation of the air conditioner 10 is acquired (S1). The control unit 80 inputs the prompt and the acquired speech data to the LLM 86, causing it to output an output sentence including control parameters used to control the air conditioner 10 (S4C, S4D). The control unit 80 acquires the results of sentiment analysis as an example of state information indicating the user's psychological state, estimated based on the input voice data or speech data acquired from the voice data (S4C, S81). The control unit 80 controls the operation of the air conditioner 10 based on the control parameters included in the output sentence and the results of the sentiment analysis (S5A, S6).

[0145] According to the control system 1 described above, it is possible to control the air conditioner 10 more flexibly by reflecting the psychological state of the user, such as emotions.

[0146] In this embodiment, the prompt includes, for example, a command statement that is an example of a command for generating output sentence D2 so that output sentence D2 includes a "sentiment" evaluation as an example of the result of the sentiment analysis and control parameters 25 used to control the air conditioner 10, as shown in prompt D1 in Fig. 6 and output sentence D2 in Fig. 7, and a constraint condition that is an example of a constraint that conditions the generation of output sentence D2. The control unit 80 obtains the result of the sentiment analysis from output sentence D2 by the LLM 86 (S4C). As a result, output sentence D2 including the result of the sentiment analysis can be obtained in a straight line from the utterance data by the LLM 86, for example.

[0147] In a modification of this embodiment, the control unit 80 causes the LLM 86 to generate an output sentence based on the results of emotion analysis processing from speech data as an example of the results of emotion analysis (S81, S4D). As a result, an output sentence that reflects the user's emotion can be obtained even from the results of emotion analysis performed by a device other than the LLM 86.

[0148] In this embodiment, the LLM 86 generates an output sentence in which the control parameters have been changed in accordance with the utterance data and the results of the sentiment analysis (S4C, S4D). The storage unit 81 stores the amount of change in the control parameters in accordance with the results of the sentiment analysis, in association with the state (e.g., positive or negative) indicated by the results of the sentiment analysis, as shown in the individual setting table T1 in FIG. 5A, for example. This allows the results of the sentiment analysis to be reflected in the amount of change in the control parameters.

[0149] In this embodiment, the LLM 86 changes the control parameters using at least the first candidate of the first candidate of the control parameters generated in response to the speech data and the second candidate of the control parameters generated in response to the results of the emotion analysis. This allows the change of the control parameters in response to the content of the speech to be prioritized as the first candidate, for example.

[0150] In this embodiment, the control system 1 provides an information processing method for controlling the air conditioner 10 (S1 to S6). The method is executed by the control unit 80 of the server 8 (an example of one or more computers). A prompt to be input to the LLM 86, which is an example of a large-scale language model, is stored in the memory unit 81 of the server 8. The LLM 86 is trained by machine learning to generate an output sentence in response to an input sentence. The method includes the control unit 80 acquiring, from voice data input via the microphone 75 (an example of an input interface) of the terminal device 7 (an example of one or more computers), speech data of an example of text related to the operation of the air conditioner 10, which is based on the input voice data (S1). The method also includes the control unit 80 inputting the prompt and the acquired speech data into the LLM 86, and causing the LLM 86 to output an output sentence including a control parameter used to control the air conditioner 10 (S4C, S4D). This method includes the control unit 80 acquiring the result of emotion analysis as an example of state information indicating the user's psychological state, estimated based on the input voice data or speech data acquired from the voice data (S4C, S81). This method also includes the control unit 80 controlling the operation of the air conditioner 10 based on the control parameters included in the output sentence and the result of the emotion analysis (S5A, S6).

[0151] In this embodiment, a control program 88 is provided as an example of a program for causing the control unit 80 to execute the above-described information processing method. According to the above-described information processing method and program, it is possible to generate control commands for the air conditioner 10 that reflect the psychological state, such as the emotions expressed in the user's speech, making it easier for the user to flexibly control the air conditioner 10.

[0152] (Other embodiments) Although the present disclosure has been described above with reference to Embodiments 1 to 3, the present disclosure is not limited to the above-described embodiments. The technology in the present disclosure is not limited to these embodiments and can be applied to embodiments in which appropriate modifications, substitutions, additions, omissions, etc. are made.

[0153] For example, in the above third embodiment, the user may be able to select whether or not to reflect the emotion analysis results in the control command to the air conditioner 10 when controlling the air conditioner 10 or when setting it up as in the second embodiment.

[0154] In each of the above embodiments, an example has been described in which the LLM 86, the prompt DB 87, etc. are stored in the memory unit 81 of the server 8 in the control system 1. In the present embodiment, the LLM 86 and / or the prompt DB 87 may be stored in the memory unit 71 of the terminal device 7, or in the memory unit 11 of the air conditioner 10. The information processing method of the present disclosure may be executed in the terminal device 7 or the air conditioner 10.

[0155] In the above-described embodiments, electronic devices such as smartphones, tablet terminals, and PCs have been described as examples of the terminal device 7. For example, the terminal device 7 may be configured by a wearable device having a microphone or the like, or a remote controller for the air conditioner 10, or the indoor unit 20 may be equipped with the same functions as the terminal device 7 described above.

[0156] In each of the above embodiments, an example has been described in which utterance data converted from voice data to text data is input to the LLM 86. The text data input to the LLM 86 is not limited to utterance data, and may be generated in response to, for example, a user operation on the operation unit 73 of the terminal device 7. The operation unit 73 of the present embodiment is an example of an input interface in the control system 1. In each of the above embodiments, the input interface of the control system 1 inputs text data or voice data indicating the text of the utterance.

[0157] In the above embodiments, examples have been described in which the air conditioner 10 is equipped with an indoor temperature sensor 14. The air conditioner 10 may also have various sensors, such as an infrared camera. For example, in away mode, the control parameters of the air conditioner 10 may be determined based on sensor information from the infrared camera in addition to speech data.

[0158] In the above embodiments, examples have been described in which the user's psychology is evaluated by emotion analysis, and the positive or negative analysis result is reflected in the control of the air conditioner 10. The psychology may be evaluated using various binary classifications, such as "good mood" or "bad mood," "in a hurry" or "not in a hurry," and "angry" or "happy," or may be evaluated using three or more classifications. The psychology may also be evaluated by predicting a predicted value indicating a predetermined index as a continuous value.

[0159] Although the present disclosure has been fully described in connection with the preferred embodiments with reference to the accompanying drawings, various changes and modifications will be apparent to those skilled in the art, and such changes and modifications are to be understood as included within the scope of the present disclosure as defined by the appended claims unless they depart therefrom.

[0160] Furthermore, the general and specific aspects of the present disclosure may be realized by a system, a method, a computer program, and a computer-readable storage medium, as well as combinations thereof.

[0161] Additionally, terms such as "first," "second," etc. are used herein for descriptive purposes only and should not be understood as expressing or implying the relative importance or ranking of technical features. Features qualified as "first" and "second" expressly or imply the inclusion of one or more of such features.

[0162] Aspects of the present disclosure The following describes exemplary aspects of the present disclosure.

[0163] A first aspect of the present disclosure is a control system for an air conditioner, A control unit; an input interface for inputting data; a memory unit for storing prompts to be input to the large-scale language model; Large-scale language models are trained using machine learning to generate output sentences based on input sentences. The control unit acquiring, from the data input via the input interface, text based on the input data, the text relating to the operation of the air conditioner; The prompt and the acquired text are input to a large-scale language model to generate an output sentence including control parameters used to control the air conditioner; The operation of the air conditioner is controlled based on the control parameters included in the output statement.

[0164] In a second aspect, in the control system of the first aspect, The storage unit stores prompts to be input to the large-scale language model in association with each of a plurality of users; The control unit Associating text with each user of a plurality of users and retrieving the text; A prompt corresponding to the user associated with the retrieved text is input to the large-scale language model from among the prompts stored in the memory unit. 10. The control system of claim 1.

[0165] In a third aspect, in the control system of the first or second aspect, The prompts include instructions to the large-scale language model to generate output sentences that include control parameters and constraints that condition the generation of the output sentences.

[0166] In a fourth aspect, in the control system of any one of the first to third aspects, The storage unit stores a history that associates the acquired text with an output sentence generated from the text by a large-scale language model; The control unit further inputs the history stored in the storage unit into the large-scale language model to generate an output sentence. 10. The control system of claim 1.

[0167] In a fifth aspect, in the control system of any one of the first to fourth aspects, The control unit Based on the acquired text, additional information related to the text is acquired; Additional information is input to the large-scale language model to generate output sentences. 10. The control system of claim 1.

[0168] In a sixth aspect, in the control system of any one of the first to fifth aspects, The input interface receives text data or voice data representing a text.

[0169] In a seventh aspect, The control system according to any one of the first to sixth aspects comprises: an information processing device; A terminal device; an air conditioner; The information processing device, the terminal device, and the air conditioner are connected to each other so as to be able to perform data communication.

[0170] An eighth aspect is an information processing device in the control system of the seventh aspect, A control unit; It has a communication interface for data communication with external devices.

[0171] A ninth aspect is a terminal device in the control system of the seventh aspect, an input interface; It has a communication interface for data communication with external devices.

[0172] A tenth aspect is an air conditioner in the control system of the seventh aspect, a storage unit that stores setting values ​​used to control the air conditioner; It has a communication interface for data communication with external devices.

[0173] An eleventh aspect is an information processing method for controlling an air conditioner, The information processing method is executed by a control unit of one or more computers, a memory unit of one or more computers storing prompts to be input to the large-scale language model; Large-scale language models are trained using machine learning to generate output sentences based on input sentences. The information processing method is performed by the control unit. obtaining, from data input via one or more computer input interfaces, text based on the input data, the text relating to the operation of the air conditioner; inputting the prompt and the acquired text into a large-scale language model to output an output sentence including control parameters used to control the air conditioner; and This includes controlling the operation of the air conditioner based on the control parameters included in the output statement.

[0174] A twelfth aspect is a program for causing a control unit to execute the information processing method of the eleventh aspect. [Industrial Applicability]

[0175] The present disclosure is applicable to various control systems that control air conditioners using large-scale language models. [Explanation of symbols]

[0176] 1. Control System 10 Air conditioner 11, 71, 81 Storage section 12, 70, 80 Control section 13, 72, 82 Communications Department 20 Indoor unit 30 Outdoor unit 73 Operation section 74 Display section 75 Mike 76 Speaker 86 LLM

Claims

1. An air conditioner control system, A control unit; an input interface for inputting data; a memory unit for storing prompts to be input to the large-scale language model; the large-scale language model is trained by machine learning to generate output sentences in response to input sentences; The control unit acquiring, from the data input via the input interface, text based on the input data, the text being related to operation of the air conditioner; inputting the prompt and the acquired text into the large-scale language model to generate an output sentence including a control parameter used to control the air conditioner; Controlling the operation of the air conditioner based on the control parameters included in the output statement Control system.

2. the storage unit stores prompts to be input to the large-scale language model in association with each of a plurality of users; The control unit obtaining the text in association with each user of the plurality of users; inputting a prompt corresponding to the user associated with the acquired text from among the prompts stored in the storage unit into the large-scale language model; The control system of claim 1 .

3. The prompt includes an instruction to cause the large-scale language model to generate an output sentence that includes the control parameter, and a constraint that conditions the generation of the output sentence. The control system of claim 1 .

4. the storage unit stores a history associating the acquired text with the output sentence generated from the text by the large-scale language model; The control unit further inputs the history stored in the storage unit into the large-scale language model to generate the output sentence. The control system of claim 1 .

5. The control unit Based on the acquired text, additional information related to the text is acquired; The additional information is further input to the large-scale language model to generate the output sentence. The control system of claim 1 .

6. The input interface inputs text data or voice data representing the text. The control system of claim 1 .

7. The control system according to any one of claims 1 to 6, an information processing device; A terminal device; The air conditioner, The information processing device, the terminal device, and the air conditioner are connected to each other so as to be able to communicate data. Control system.

8. An information processing device in the control system according to claim 7, the control unit; A communication interface for data communication with an external device is provided. Information processing device.

9. A terminal device in the control system according to claim 7, the input interface; A communication interface for data communication with an external device is provided. Terminal device.

10. The air conditioner in the control system according to claim 7, a storage unit that stores setting values ​​used to control the air conditioner; A communication interface for data communication with an external device is provided. Air conditioner.

11. An information processing method for controlling an air conditioner, comprising: The information processing method is executed by a control unit of one or more computers, a memory unit of the one or more computers stores prompts to be input to a large-scale language model; the large-scale language model is trained by machine learning to generate output sentences in response to input sentences; The information processing method includes, by the control unit: acquiring, from the data input via the one or more computer input interfaces, text based on the input data, the text relating to operation of the air conditioner; inputting the prompt and the acquired text into the large-scale language model, and outputting an output sentence including a control parameter used to control the air conditioner; and Controlling the operation of the air conditioner based on the control parameters included in the output statement. An information processing method, including:

12. A program for causing the control unit to execute the information processing method according to claim 11.

Images