Air conditioner control method, computer program, and air conditioner control device

Abstract

The present invention provides an air conditioner control method, a computer program, and an air conditioning control device that make an instruction to an air conditioner into an appropriate command. The air conditioner control method causes a computer to execute processing for: receiving an input sentence relating to an air conditioner; generating, using a generative model, a control command to the air conditioner corresponding to the input sentence when the input sentence concerns control of the air conditioner, and a notification that control corresponding to the input sentence cannot be performed when the input sentence does not concern control of the air conditioner; and transmitting the generated control command to the air conditioner or outputting the generated notification.

Description

Air conditioner control method, computer program, and air conditioner control apparatus

[0001] The present disclosure relates to an air conditioner control method, a computer program, and an air conditioner control apparatus.

[0002] Patent literature discloses a technology that allows a user to give instructions to an air conditioner via voice. In addition to voice, technologies that accept instructions to an air conditioner in natural language using generative AI have also been implemented.

[0003] Japanese Unexamined Patent Publication No. Hei 9-14722

[0004] An object of the present disclosure is to provide an air conditioner control method, a computer program, and an air conditioner control apparatus that convert an instruction to an air conditioner into an appropriate command.

[0005] An air conditioner control method according to one embodiment receives an input sentence relating to an air conditioner, and when the input sentence corresponds to control of the air conditioner, generates a control command for the air conditioner according to the input sentence, and when the input sentence does not correspond to control of the air conditioner, generates a notification indicating that control is not possible according to the input sentence, using a generative model, and causes a computer to execute a process of transmitting the generated control command to the air conditioner or outputting the generated notification.

[0006] The air conditioner control method according to one embodiment may cause the computer to execute a process of generating the notification so as to include a reason why the input sentence does not correspond to control of the air conditioner.

[0007] The air conditioner control method according to one embodiment may cause the computer to execute a process of adding alternative proposal information for the input sentence to the control-impossible notification and outputting the same.

[0008] The air conditioner control method according to one embodiment stores the received input sentence and the generated control command or control-impossible notification as a history, and causes the computer to execute a process of generating the control command or the control-impossible notification for a new input sentence using the generative model and the history.

[0009] In one embodiment of the air conditioner control method, the computer may be instructed to perform a process to infer the state of the user who input the input statement, based on the difference between the control command corresponding to the new input statement and the control command for the air conditioner in the history.

[0010] In one embodiment of the air conditioner control method, if an abnormal state is inferred from the user's state, the computer may be instructed to execute a process to stop accepting input messages from the user.

[0011] In one embodiment of the air conditioner control method, if the input statement does not correspond to the control of the air conditioner, the computer may be instructed to perform a process to generate an utterance related to the input statement.

[0012] In one embodiment of the air conditioner control method, the input statement may include information about a lighting device or sound equipment in addition to the air conditioner.

[0013] In one embodiment of the air conditioner control method, information on the temperature, humidity, or weather inside and outside the space in which the air conditioner is installed is provided to the generation model, and the computer is instructed to use the generation model to perform a process of generating a control command to the air conditioner or a notification that control is not possible.

[0014] In one embodiment of the air conditioner control method, the computer may be instructed to perform a process of providing the generation model with a prompt that instructs the computer to generate a control command to the air conditioner corresponding to the input statement if the input statement corresponds to the control of the air conditioner, and to generate a notification that control is not possible according to the input statement if the input statement does not correspond to the control of the air conditioner, and the input statement itself.

[0015] In one embodiment of the air conditioner control method, the computer may be instructed to perform a process of providing the generation model with a document including a description of the air conditioner, along with the prompt and the input statement.

[0016] One embodiment of the computer program causes a computer connected to an air conditioner to receive an input statement relating to the air conditioner, and, if the input statement corresponds to the control of the air conditioner, to generate a control command to the air conditioner corresponding to the input statement, or if the input statement does not correspond to the control of the air conditioner, to generate a notification of control failure corresponding to the input statement, using a generation model, and then to transmit the generated control command to the air conditioner or output the generated notification.

[0017] An air conditioner control device according to one embodiment includes a processing unit that receives an input statement relating to an air conditioner, generates a control command to the air conditioner corresponding to the input statement if the input statement corresponds to the control of the air conditioner, or generates a notification of control failure corresponding to the input statement if the input statement does not correspond to the control of the air conditioner, using a generation model, and then transmits the generated control command to the air conditioner or outputs the generated notification.

[0018] According to one embodiment of the air conditioner control method, computer program, and air conditioner control device, if the instruction from the user to the air conditioner is appropriate, a command is issued, and if it is inappropriate, the user is automatically notified of this.

[0019] This is a schematic diagram showing the overall configuration of the air conditioning system according to the first embodiment. This is a block diagram showing the configuration of the control system of the air conditioner. This is a block diagram showing the configuration of the control system of the user terminal. This is a block diagram showing the configuration of the control system of the first server. This is a block diagram showing the configuration of the control system of the second server. This is a functional block diagram showing the functions of the user terminal in the air conditioning system. This is a flowchart showing an example of an air conditioning control procedure by the air conditioning system in the first embodiment. This is a diagram showing an example of a generated prompt. This is a diagram showing an example of a control screen output to the display unit. This is a diagram showing an example of a control screen output to the display unit. This is a diagram showing an example of a control screen output to the display unit. This is a diagram showing an example of a control screen output to the display unit. This is a diagram showing an example of a control screen output to the display unit. This is a flowchart showing an example of an air conditioning processing procedure by the air conditioning system in the second embodiment. This is a flowchart showing an example of an air conditioning processing procedure by the air conditioning system in the second embodiment. This is a functional block diagram showing the functions of the user terminal in the air conditioning system in the third embodiment. This is a flowchart showing an example of an air conditioning control procedure by the air conditioning system in the third embodiment. This is a flowchart showing an example of an air conditioning control procedure by the air conditioning system in the third embodiment. This shows an example of notification of an abnormal state on the control screen. This shows an example of display of spoken text on the control screen.

[0020] A specific example of the air conditioner control method according to this embodiment will be described below with reference to the drawings. However, this technology is not limited to these disclosures, and is as indicated by the claims, with all modifications intended to be within the meaning and scope of the claims.

[0021] (First Embodiment) Figure 1 is a schematic diagram showing the overall configuration of the air conditioning system 1 according to the first embodiment. The air conditioning system 1 according to the first embodiment comprises an air conditioner 100 and a user terminal 200. The air conditioner 100 and the user terminal 200 are connected to each other via a local communication network NW1 such as Wi-Fi®.

[0022] The air conditioner 100 is a household or commercial air conditioner for adjusting the temperature, humidity, air quality, etc., of a room, and comprises an indoor unit and an outdoor unit. The air conditioner 100 adjusts the temperature, humidity, etc., of a room by operating in accordance with control commands output from a remote controller 150 or a user terminal 200. In the first embodiment, the air conditioner 100 is described below as a so-called air conditioner comprising an indoor unit and an outdoor unit, but "air conditioner" means a device that adjusts the indoor environment (humidity, temperature, air quality), and encompasses the air conditioner 100, air purifier 151, circulator 152, diffuser 153, outdoor air handling unit 156 (see Figure 6), humidifier 157 (see Figure 6), and dehumidifier 158 (see Figure 6).

[0023] The user terminal 200 is a device such as a smartphone, tablet, personal computer, or smart speaker operated by the user of the air conditioner 100. The user terminal 200 is equipped with artificial intelligence (AI). The artificial intelligence installed in the user terminal 200 is constructed by a generative model MD2 that generates new content in response to inputs such as text, voice, still images, and videos. The generative model MD2 may be a proprietary model that has been fine-tuned and / or distilled (lightened) for a specific application. The content generated by the generative model MD2 includes text, sound such as voice or music, and images such as still images or videos. The user terminal 200 can create the indoor environment desired by the user by using the generative AI to interact with the user, generate control commands for the air conditioner 100, and control the operation of the air conditioner 100 based on the generated control commands.

[0024] In the example shown in Figure 1, there is one user terminal 200, but there may be multiple user terminals 200 capable of controlling the air conditioner 100.

[0025] The air conditioning system 1 may further include an air purifier 151, a circulator 152, a diffuser 153, lighting equipment 154, and sound equipment 155. The air purifier 151 is a device for purifying the indoor air and providing clean air. The circulator 152 is a device for circulating the indoor air. The diffuser 153 is a device for diffusing fragrance. The lighting equipment 154 is a device for illuminating the indoor space. The sound equipment 155 is a device for playing music, ambient sounds, etc. These devices 151 to 155 are connected to at least one of the air conditioner 100 and the user terminal 200 via a communication network NW1 so as to be able to communicate.

[0026] The air conditioning system 1 may further include a wearable device 250 worn by the user. The wearable device 250 measures, for example, at least one of the user's heart rate, body temperature, sweating, blood pressure, respiration, brain waves, and body composition, and outputs the measurement results as the user's biometric information. The output destination may be the air conditioning unit 100 or the user terminal 200.

[0027] The air conditioning system 1 may further include a first server 310 or a second server 320 accessible from the air conditioner 100 or a user terminal 200. The air conditioner 100 or the user terminal 200 and the first server 310 or the second server 320 are connected to each other via an external communication network NW2, such as the Internet.

[0028] The first server 310 is equipped with a generative AI. The generative AI of the first server 310 is constructed by a generative model MD3 that generates new content in response to inputs such as text, audio, still images, and videos. The generative model MD3 may include existing models such as GPT (Generative Pretrained Transformer, registered trademark), BERT (Bidirectional Encoder Representations from Transformers), GAN (Generative Adversarial Network), WaveNet, DALL-E, and StableDiffusion, or it may be a unique model that has been fine-tuned for a specific application from an existing model. The first server 310 generates content such as text, audio, still images, and videos in response to a request from the air conditioner 100 or user terminal 200, and returns the generated content to the requester.

[0029] The second server 320 is a search server that searches for relevant documents or information in response to an input query and returns the relevant documents or information to the requester. Existing search engines such as vector search are used to search for relevant documents and information. In one example, the second server 320 is a general-purpose search server that returns search results in response to external search requests. Alternatively, the second server 320 may be a dedicated search server that can access a database DB that stores information such as operation manuals, service guides, technical handbooks, installation instructions, customer information, operation data during trial operation, and operation data after the start of operation for the air conditioner 100, and searches the database DB in response to external search requests and returns the search results to the requester. The database DB may also register construction information when the air conditioner 100 was installed (installation location of indoor and outdoor units, piping length, floor plan, year of construction), information about the installed air conditioner 100 (model, Ua value, horsepower, etc.), and sales information for each employee.

[0030] The air conditioning system 1 according to the first embodiment realizes "anomaly response by generation AI" by utilizing at least one of the generation model MD2 installed on the user terminal 200 and the generation model MD3 installed on the first server 310. This "anomaly response by generation AI" may be realized using only the content output from the generation model MD2 installed on the user terminal 200, or it may be realized using only the content output from the generation model MD3 installed on the first server 310.

[0031] Furthermore, the above-mentioned "anomaly response by generation AI" may be realized through cooperation between the generation model MD2 installed on the user terminal 200 and the generation model MD3 installed on the first server 310.

[0032] Furthermore, the above-mentioned "anomaly response by generation AI" may be implemented using a RAG (Retrieval-Augmented Generation) mechanism. That is, in response to an input query from the air conditioner 100 or user terminal 200, the second server 320 may output documents or information related to the input query, and the first server 310 may generate content based on the documents or information output from the second server 320 and return the content to the air conditioner 100 or user terminal 200. By using a RAG mechanism, more accurate and relevant content can be provided.

[0033] In the first embodiment, the user terminal 200 is configured to have the generation model MD2 installed. Alternatively, the generation model may be installed in the air conditioner 100. In this case, the generation model MD1 (see Figure 2) installed in the air conditioner 100 may be used to realize the "anomaly response by generation AI" described above. Alternatively, the generation model MD1 of the air conditioner 100 and the generation model MD3 of the first server 310 may cooperate to realize the "anomaly response by generation AI" described above.

[0034] Furthermore, both the air conditioner 100 and the user terminal 200 may be equipped with generation models. In this case, at least two of the generation models, MD1 of the air conditioner 100, MD2 of the user terminal 200, and MD3 of the first server 310, may work together to realize the "anomaly response by generation AI" described above.

[0035] Figure 2 is a block diagram showing the configuration of the control system of the air conditioner 100. The air conditioner 100 comprises a control unit 101, a storage unit 102, a communication unit 103, a sensor unit 104, and a drive unit 110.

[0036] The control unit 101 includes a CPU (Central Processing Unit), ROM (Read Only Memory), RAM (Random Access Memory), and the like. The ROM in the control unit 101 stores control programs for controlling the operation of the drive unit 110. The CPU in the control unit 101 executes the control programs stored in the ROM and various computer programs stored in the memory unit 102, and controls the operation of the drive unit 110, thereby making the entire device function as the air conditioner 100 in this disclosure.

[0037] The storage unit 102 is equipped with a storage device such as flash memory. The storage unit 102 stores various computer programs to be executed by the control unit 101, as well as data necessary for the execution of these programs.

[0038] The computer program (program product) stored in the memory unit 102 includes a control program PG1 for realizing the "abnormal response by generated AI" described above. The control program PG1 may be a single computer program or a group of programs composed of multiple computer programs. The control program PG1 may be executed collaboratively by multiple computers.

[0039] The computer program, including the control program PG1, is provided, for example, on a non-temporary recording medium RM1 on which the computer program is recorded in a readable format. The recording medium RM1 is a portable memory such as a CD-ROM, USB memory, SD card, microSD card, or CompactFlash®. The control unit 101 reads various computer programs from the recording medium RM1 using a reading device (not shown in the figure) and stores the read computer programs in the storage unit 102. The computer programs stored in the storage unit 102 may also be provided via communication. In this case, the control unit 101 acquires the computer programs via communication through the communication unit 103 and stores the acquired computer programs in the storage unit 102.

[0040] The data stored in the memory unit 102 may include at least one of the following: an operation manual, a service guide, a technical handbook, installation instructions, customer information, operational data during commissioning, and operational data after the start of operation.

[0041] When the air conditioner 100 is equipped with a generation AI, the generation model MD1 that constructs the generation AI is stored in the memory unit 102. The generation model MD1 is a model that generates new content in response to inputs such as text, voice, still images, and videos. The content generated by the generation model MD1 includes text, voice or music, images such as still images or videos, etc. The generation model MD1 is a unique model that has been fine-tuned and / or distilled (lightened) for a specific application from existing models such as GPT, BERT, GAN, WaveNet, DALL-E, and StableDiffusion.

[0042] The communication unit 103 includes a communication interface for communicating with a user terminal 200 and the like. As the communication interface included in the communication unit 103, a communication interface compliant with a wireless LAN (Local Area Network) communication standard such as Wi-Fi (registered trademark) can be used. Alternatively, a communication interface compliant with a short-range wireless communication standard such as Bluetooth may be used. In addition to the user terminal 200, the communication unit 103 may be configured to be capable of communicating with an air purifier 151, a circulator 152, a diffuser 153, a lighting device 154, an acoustic device 155, and the like.

[0043] The communication unit 103 may further include a communication interface for communicating with a first server 310 or a second server 320. As such a communication interface, wireless communication interfaces such as Wi-Fi (registered trademark), 3G, 4G, 5G, and LTE (Long Term Evolution) can be used.

[0044] The sensor unit 104 includes one or more sensors that measure various physical quantities in an indoor environment or the state of a user. The sensors included in the sensor unit 104 include sensors that measure at least one of indoor temperature, humidity, heat, CO₂ concentration, and illuminance. The sensor unit 104 may include sensors that measure the user's movement, facial expression, and the like.

[0045] The drive unit 110 includes a compressor 111, a four-way switching valve 112, an indoor fan 113, a flap 114, and the like. The compressor 111 is a device for sucking in a refrigerant in a gaseous state of constant temperature and low pressure, and compressing the refrigerant into a high-temperature and high-pressure gas. During cooling operation, the refrigerant compressed by the compressor 111 returns to the suction side of the compressor 111 via an outdoor heat exchanger, an expansion valve, a first indoor heat exchanger, a solenoid valve, and a second indoor heat exchanger. In this refrigeration cycle, heat is dissipated by the outdoor heat exchanger functioning as a condenser, and indoor air is cooled by the first and second indoor heat exchangers functioning as evaporators to perform cooling. During heating operation, the path is switched by the four-way switching valve 112, and heating is performed in a cycle reverse to that during cooling operation.

[0046] The indoor fan 113 is a device for uniformly circulating cooled air or heated air throughout the entire room. The airflow volume of the indoor fan 113 is controlled by a control command from the control unit 101. By adjusting the airflow volume of the indoor fan 113, the efficiency of cooling and heating can be improved, and energy consumption can be reduced.

[0047] The flap 114 is a device for adjusting the direction of air blown out from the air conditioner 100. The orientation of the flap 114 is controlled by a control command from the control unit 101. By securing an appropriate air flow with the flap 114, the efficiency of cooling and heating can be improved, and energy consumption can be reduced.

[0048] Figure 3 is a block diagram showing the configuration of a control system of the user terminal 200. The user terminal 200 is a terminal device such as a smartphone, a tablet terminal, a personal computer, or a smart speaker, and includes a control unit 201, a storage unit 202, a communication unit 203, an imaging unit 204, a voice input unit 205, a voice output unit 206, a positioning unit 207, an operation unit 208, a display unit 209, and the like.

[0049] The control unit 201 includes a CPU, a ROM, a RAM, and the like. The ROM included in the control unit 201 stores a control program and the like for controlling the operation of each hardware unit included in the user terminal 200. The CPU in the control unit 201 reads and executes the control program stored in the ROM and the computer program stored in the storage unit 202, and controls the operation of each of the above hardware units, thereby causing the entire device to function as the user terminal 200 in the present disclosure. The RAM included in the control unit 201 temporarily stores data used during execution of calculations.

[0050] In the first embodiment, the control unit 201 is configured to include a CPU, ROM, and RAM. Alternatively, the control unit 201 may be one or more control circuits or processing circuits that include, for example, a GPU (Graphics Processing Unit), FPGA (Field Programmable Gate Array), DSP (Digital Signal Processor), quantum processor, volatile or non-volatile memory, etc. The control unit 201 may also include functions such as a clock that outputs date and time information, a timer that measures the elapsed time from the time a measurement start instruction is given to the time a measurement end instruction is given, and a counter that counts numbers.

[0051] The storage unit 202 includes a storage device such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive). The storage unit 202 stores various computer programs executed by the control unit 201 and various data used by the control unit 201.

[0052] The computer program (program product) stored in the memory unit 202 includes a control program PG2 for realizing the "anomaly response by generating AI" described above. The control program PG2 may be a single computer program or a group of programs composed of multiple computer programs. The control program PG2 may be executed collaboratively by multiple computers. For example, the "anomaly response by generating AI" described above may be realized by the collaboration of the control program PG1 installed in the air conditioner 100 and the control program PG2 installed in the user terminal 200.

[0053] The computer program, including the control program PG2, is provided, for example, on a non-temporary recording medium RM2 on which the computer program is recorded in a readable format. The recording medium RM2 is a portable memory such as a CD-ROM, USB memory, SD card, microSD card, or CompactFlash®. The control unit 201 reads various computer programs from the recording medium RM2 using a reading device (not shown in the figure) and stores the read computer programs in the storage unit 202. The computer programs stored in the storage unit 202 may also be provided via communication. In this case, the control unit 201 acquires the computer programs via communication through the communication unit 203 and stores the acquired computer programs in the storage unit 202.

[0054] The memory unit 202 includes a generation model MD2. The generation model MD2 is a model that generates new content in response to inputs such as text, audio, still images, and videos. The content generated by the generation model MD2 includes text, audio or music, images such as still images or videos, etc. The generation model MD2 is a unique model that has been distilled (lightened) for a specific purpose from existing models such as GPT, BERT, GAN, WaveNet, DALL-E, and StableDiffusion.

[0055] The communication unit 203 is equipped with a communication interface for communicating with the air conditioner 100 and the like. The communication interface provided by the communication unit 203 can be a communication interface compliant with a wireless LAN communication standard such as Wi-Fi (registered trademark). Alternatively, a communication interface compliant with a short-range wireless communication standard such as Bluetooth may be used. In addition to the air conditioner 100, the communication unit 203 may be configured to communicate with an air purifier 151, a circulator 152, a diffuser 153, lighting equipment 154, or audio equipment 155, etc.

[0056] The communication unit 203 further includes a communication interface for communicating with the first server 310 or the second server 320. Such a communication interface can be a wireless communication interface such as Wi-Fi®, 3G, 4G, 5G, or LTE (Long Term Evolution).

[0057] The imaging unit 204 is equipped with an image sensor such as a CMOS (Complementary Metal Oxide Semiconductor) or a CCD (Charge-Coupled Device), and generates an image (still image or video) of the object to be imaged by imaging the object. The imaging unit 204 outputs the image data related to the generated image to the control unit 201. The control unit 201 stores the input image data in the storage unit 202. The control unit 201 may display the image based on the image data on the display unit 209, or transmit it to the outside via the communication unit 203.

[0058] The audio input unit 205 includes a microphone for collecting sound, a processing circuit for converting the collected sound into a digital signal (audio data), and the like. The audio data generated in the audio input unit 205 is sent to the control unit 201, where appropriate processing such as noise reduction is performed. The audio data generated in the audio input unit 205 is also stored in the storage unit 202 or transmitted to the first server 310 via the communication unit 203.

[0059] The audio output unit 206 is equipped with a speaker that outputs sound. The audio output unit 206 plays sound based on the audio data provided by the control unit 201. In the first embodiment, the audio input unit 205 and the audio output unit 206 are configured to input and output sound, but sounds other than voice (ambient sounds, music, etc.) may be input to the audio input unit 205 and sounds other than voice (ambient sounds, music, etc.) may be output from the audio output unit 206.

[0060] The positioning unit 207, for example, is equipped with a GPS (Global Positioning System) receiver and receives radio waves transmitted from GPS satellites to determine the current location of the user terminal 200. The positioning unit 207 outputs location information related to the determined current location of the user terminal 200 to the control unit 201.

[0061] The operation unit 208 is equipped with operating devices such as a touch panel, keyboard, and switches, and accepts various inputs and operations from the user. The control unit 201 acquires information input through the operation unit 208 and performs appropriate control based on the various operation information provided by the operation unit 208.

[0062] The display unit 209 is equipped with a display device such as a liquid crystal monitor or an organic EL (Electro-Luminescence) monitor, and displays information that should be notified to the user in response to instructions from the control unit 201.

[0063] Figure 4 is a block diagram showing the configuration of the control system of the first server 310. The first server 310 is a dedicated or general-purpose server computer and includes a control unit 311, a storage unit 312, a communication unit 313, and the like.

[0064] The control unit 311 includes a CPU, ROM, RAM, etc. The ROM in the control unit 311 stores control programs and the like that control the operation of each hardware component of the first server 310. The CPU in the control unit 311 reads and executes the control programs stored in the ROM and the computer programs stored in the memory unit 312, and controls the operation of each hardware component, thereby making the entire device function as the first server 310 in this disclosure. The RAM in the control unit 311 temporarily stores data used during the execution of calculations.

[0065] In the first embodiment, the control unit 311 is configured to include a CPU, ROM, and RAM, but the configuration of the control unit 311 is not limited to the above. The control unit 311 may also include functions such as a clock for outputting date and time information, a timer for measuring the elapsed time from the time a measurement start instruction is given until a measurement end instruction is given, and a counter for counting numbers.

[0066] The storage unit 312 includes a storage device such as an HDD or SSD. The storage unit 312 stores various computer programs executed by the control unit 311, various data acquired through the communication unit 313, and various data used by the control unit 311.

[0067] The computer program (program product) stored in the memory unit 312 may include a control program PG3 for realizing the "abnormal response by generated AI" described above. The control program PG3 may be a single computer program or a group of programs composed of multiple computer programs. The control program PG3 may be executed collaboratively by multiple computers. For example, the control program PG3 may realize the "abnormal response by generated AI" described above by collaborating with the control program PG1 installed in the air conditioner 100 or the control program PG2 installed in the user terminal 200.

[0068] The computer program, including the control program PG3, is provided, for example, on a non-temporary recording medium RM3 on which the computer program is recorded in a readable format. The recording medium RM3 is a portable memory such as a CD-ROM, USB memory, SD card, microSD card, or CompactFlash®. The control unit 311 reads various computer programs from the recording medium RM3 using a reading device (not shown in the figure) and stores the read computer programs in the storage unit 312. The computer programs stored in the storage unit 312 may also be provided via communication. In this case, the control unit 311 acquires the computer programs via communication through the communication unit 313 and stores the acquired computer programs in the storage unit 312.

[0069] The memory unit 312 may include a generation model MD3. The generation model MD3 is a model that generates new content in response to inputs such as text, audio, still images, and videos. The content generated by the generation model MD3 includes text, audio or music, images such as still images or videos, etc. The generation model MD3 may be an existing model such as GPT, BERT, GAN, WaveNet, DALL-E, or StableDiffusion, or it may be a proprietary model fine-tuned for a specific application.

[0070] The communication unit 313 is equipped with a communication interface for communicating with a user terminal 200 or the like. The communication interface provided by the communication unit 313 can be a wireless communication interface such as 3G, 4G, 5G, or LTE (Long Term Evolution).

[0071] The first server 310 may include an operation unit for receiving operations from administrators, a display unit for displaying information to be notified to administrators, etc. The first server 310 may be a single computer, or it may be a computer system composed of multiple computers and peripheral devices.

[0072] Figure 5 is a block diagram showing the configuration of the control system of the second server 320. The second server 320 is a dedicated or general-purpose server computer and includes a control unit 321, a storage unit 322, a communication unit 323, and the like.

[0073] The control unit 321 includes a CPU, ROM, RAM, etc. The ROM in the control unit 321 stores control programs and the like that control the operation of each hardware component of the second server 320. The CPU in the control unit 321 reads and executes the control programs stored in the ROM and the computer programs stored in the memory unit 322, and controls the operation of each hardware component, thereby making the entire device function as the second server 320 in this disclosure. The RAM in the control unit 321 temporarily stores data used during the execution of calculations.

[0074] In the first embodiment, the control unit 321 is configured to include a CPU, ROM, and RAM, but the configuration of the control unit 321 is not limited to the above. The control unit 321 may also include functions such as a clock for outputting date and time information, a timer for measuring the elapsed time from the time a measurement start instruction is given until a measurement end instruction is given, and a counter for counting numbers.

[0075] The storage unit 322 includes a storage device such as an HDD or SSD. The storage unit 322 stores various computer programs executed by the control unit 321, various data acquired through the communication unit 323, and various data used by the control unit 321.

[0076] The storage unit 322 may also include a database DB for storing information such as the operation manual, service guide, technical handbook, installation instructions, customer information, operation data during trial operation, and operation data after the start of operation for the air conditioner 100. The database DB may further register construction information when the air conditioner 100 was installed (installation location of indoor and outdoor units, piping length, floor plan, and year of construction), information about the installed air conditioner 100 (model, Ua value, horsepower, etc.), and sales information for each employee.

[0077] The communication unit 323 is equipped with a communication interface for communicating with a user terminal 200 or the like. The communication interface provided by the communication unit 323 can be a wireless communication interface such as 3G, 4G, 5G, or LTE (Long Term Evolution).

[0078] The second server 320 may include an operation unit for receiving operations from administrators, a display unit for displaying information to be notified to administrators, etc. The second server 320 may be a single computer, or it may be a computer system composed of multiple computers and peripheral devices.

[0079] In the air conditioning system 1 configured in this way, the user terminal 200 receives instructions from the user for the air conditioner 100 and other devices that can create the spatial environment, and performs appropriate processing according to whether the instructions can be handled. Figure 6 is a functional block diagram showing the functions of the user terminal 200 in the air conditioning system 1. In the first embodiment, the control unit 201 of the user terminal 200 functions as an instruction receiving unit 211, a command / notification generation unit 212, a command transmission unit 213, and a notification output unit 214 based on the control program PG2.

[0080] The control unit 201, which functions as an instruction receiving unit 211, receives input text corresponding to instructions from the user to each device, including the air conditioner 100, via operation of the operation unit 208 or voice input to the voice input unit 205. In the first embodiment, the instruction receiving unit 211 receives text corresponding to instructions from the user as input text. The input text may be a conversation expressed in natural language describing the user's instructions to each device, including the air conditioner 100, or it may be selected keywords.

[0081] The control unit 201, which functions as a command / notification generation unit 212, uses the generation model MD2 to generate control commands or control failure notifications for each piece of equipment, including the air conditioner 100, in response to user instructions received by the instruction receiving unit 211. The command / notification generation unit 212 inputs the input text corresponding to the instructions received by the instruction receiving unit 211, along with text describing the request to the model, into the generation model MD2, and outputs the command or notification obtained from the generation model MD2. The command / notification generation unit 212 stores the types and number of pieces of equipment that affect the same space as the air conditioner 100 that is the target of the user's instructions, and creates and provides text requesting commands or notifications according to the types and number of pieces of equipment, including the air conditioner 100.

[0082] The control unit 201, which functions as a command transmission unit 213, can communicate with the air conditioner 100, air purifier 151, circulator 152, diffuser 153, lighting equipment 154, sound equipment 155, air handling unit 156, humidifier 157, and dehumidifier 158 via the communication network NW1. The command transmission unit 213 converts the commands generated by the command / notification generation unit 212 into signals or data corresponding to each target device and transmits them to each device via the communication unit 203. The command transmission unit 213 may also transmit control signals to each device via wired connections through the I / O provided in the control unit 201.

[0083] The control unit 201, which functions as a notification output unit 214, outputs a notification to the voice output unit 206 or display unit 209 when the command / notification generation unit 212 generates a notification indicating that the received instruction does not correspond to the control of the air conditioner 100.

[0084] The command transmission unit 213 and the notification output unit 214 may be incorporated into the command / notification generation unit 212, and control commands may be transmitted directly from the generation model MD2 to each device via the communication unit 203. Similarly, notifications may be output directly from the generation model MD2 to the display unit 209.

[0085] Depending on the application environment of the air conditioning system 1, it may be possible to have only the air conditioner 100 among the air conditioner 100, air purifier 151, circulator 152, diffuser 153, lighting equipment 154, sound equipment 155, air handling unit 156, humidifier 157, and dehumidifier 158. The destinations of the command transmission unit 213 shown in Figures 1 and 6 are merely examples illustrating cases where many devices are included, and as mentioned above, it is possible for only the air conditioner 100 to be able to communicate. The destinations of the command transmission unit 213 only need to include the air conditioner 100, and any combination with lighting equipment 154 or humidifier 157, etc., is acceptable.

[0086] Due to the functions of the control unit 201 of the user terminal 200 described above, the air conditioning system 1 automatically issues control commands to each device, including the air conditioner 100, if the user's instructions to each device are appropriate, and outputs a notification to the user if they are inappropriate. The above functions are basically implemented in each component of the user terminal 200, but may be implemented in cooperation with the first server 310 and the second server 320 in some cases.

[0087] Figure 7 is a flowchart showing an example of the air conditioning control procedure by the air conditioning system 1 in the first embodiment. The following processing procedure starts when the user starts the control program PG2 on the user terminal 200.

[0088] When the control unit 201 of the user terminal 200 is activated, it identifies the target air conditioner 100 and other devices as devices to be controlled and stores them in the storage unit 202 (step S201). In step S201, the control unit 201 may identify the devices by checking whether it can communicate with each device, including the air conditioner 100, each time using the communication unit 203, or it may identify them from a list of devices to be controlled that is stored in the storage unit 202 in advance according to settings by the user. In step S201, the control unit 201 may also identify the devices by reading the devices that were identified and stored during the execution of the previous process.

[0089] In step S201, when the control unit 201 checks whether communication is possible via the communication unit 203, it receives and stores data from each piece of equipment, including the air conditioner 100, indicating the type, model number, etc. In step S201, the control unit 201 may also identify the current state of the identified equipment (operating, operating mode, set temperature, etc.).

[0090] The control unit 201, through the function of the instruction receiving unit 211, receives input text from the user corresponding to instructions for the target air conditioner 100 and other equipment (step S202). In step S202, the control unit 201 receives input text from the user via the operation unit 208 or the voice input unit 205 in the form of numerical values ​​for each setting target selected from the displayed screen, or in the form of natural language conversational text, etc.

[0091] The control unit 201, using the functions of the command / notification generation unit 212, generates a prompt to be given to the generation model MD2, along with the input sentence received in step S202 and the information of the controlled device identified in step S201 (step S203). In step S203, the control unit 201 may generate the prompt by reading a prompt set in the storage unit 202, automatically generate it based on a template, or generate it using the generation model MD2.

[0092] The prompt generated in step S203 instructs the system to generate a control command to the air conditioner 100 corresponding to the input statement if the given input statement corresponds to the control of the air conditioner 100, and to generate a control failure notification if the input statement does not correspond to the control of the air conditioner 100.

[0093] In step S203, the prompt may include instructions that indicate control is impossible if the input statement corresponding to the user's instruction does not correspond to the control of the target air conditioner 100 or the equipment including the air conditioner 100. This allows the user to determine what instructions to give when they receive a control failure notification.

[0094] The prompt generated in step S203 may include an instruction to output a notification of uncontrollable operation, along with information on alternative solutions, if the input statement corresponding to the user's instruction does not correspond to the control of the target air conditioner 100 or the equipment including the air conditioner 100. This allows the user to determine what instructions to give when they receive a notification of uncontrollable operation.

[0095] The control unit 201 may, after causing the generation model MD2 to determine whether the input statement corresponds to a device capable of communication, instruct the generation model MD2 again to generate a control command or generate a notification that control is not possible. The control unit 201 may also determine whether the instruction received in step S202 corresponds to the target air conditioner 100 without using the generation model MD2. It may also determine whether the wording is related to the air conditioner 100 using a rule-based method.

[0096] The control unit 201, using the functions of the command / notification generation unit 212, provides the prompt generated in step S203, along with the instruction received in step S202 and the information of the controlled equipment identified in step S201, to the generation model MD2 (step S204). In step S204, if the control unit 201 can identify the part number or model number of the controlled equipment identified in step S201, it may provide documents such as a manual explaining the specifications of the air conditioner as knowledge. This allows the control unit 201 to accurately determine whether the instruction is appropriate for the controlled air conditioner 100 and other equipment, and output a control command or notification.

[0097] The control unit 201 receives a control command or a notification of uncontrollable state output from the generated model MD2 (step S205). The control unit 201 determines whether the output from the generated model MD2 is a control command (step S206), and if it determines that it is a control command (S206: YES), it transmits the control command to each device, including the corresponding air conditioner 100, using the function of the command transmission unit 213 (step S207). The control unit 201 displays a screen on the display unit 209 indicating that a control command has been transmitted (step S208), and terminates the process.

[0098] If the control unit 201 determines in step S206 that it is not a control command (S206: NO), the output is a notification that control is not possible, and the notification output unit 214 displays the notification on the display unit 209 (step S209).

[0099] The control unit 201 waits in response to the notification in step S209 until it receives a new input message from the user, and then returns the process to step S202.

[0100] The processing procedure shown in Figure 7 will be explained in detail with reference to a diagram illustrating a specific example. Figure 8 shows an example of a generated prompt. In step S203, the control unit 201 of the user terminal 200 assigns a role to the air conditioner 100, for example, as shown in Figure 8, and generates a prompt instructing that if the given input statement corresponds to controlling the air conditioner 100, it should generate a control command to the air conditioner 100 corresponding to the input statement, and if the input statement does not correspond to controlling the air conditioner 100, it should generate a notification that control is not possible. As described above, the prompt exemplified in Figure 8 may be one that is stored in the storage unit 202.

[0101] The prompt generated in step S203 may include, as shown in Figure 8, an additional instruction to acquire and use the data from the operation manual as knowledge. Similarly, the prompt may include instructions for the output format, such as outputting "cannot" if the input statement is something that can be instructed to the air conditioner 100, outputting the "reason" if the input statement is not something that can be instructed to the air conditioner 100, and asking the user for further instructions.

[0102] Figures 9-13 show examples of control screens 290 output to the display unit 209. The control screens 290 shown in Figures 9-13 are displayed when the user starts the control program PG2 for sending control commands to the air conditioner 100 from the user terminal 200. In all of Figures 9-13, the control screens 290 are shown after the user has entered text and the message output by the command / notification generation unit 212 of the control unit 201 has been reflected in response.

[0103] The control screen 290, as a screen for receiving instructions from the user, includes an input area 291 that accepts text input manually or by voice, similar to a messaging application, and an output area 292 that displays the input text and the output from the generation model MD2. The control screen 290 displays the text (message) input by the user and the output from the generation model MD2 in the form of speech bubbles.

[0104] In the output area 292 of the control screen 290 shown in Figure 9, the text "Turn on heating with the air conditioner set to a temperature of 28°C." is displayed in a speech bubble as input received from the user. In response, the output area 292 of the control screen 290 in Figure 9 also shows text indicating that a control command has been sent to the air conditioner by the command / notification generation unit 212 and the command transmission unit 213, stating "A message has been sent to the air conditioner to start heating operation at a set temperature of 28°C. The fan speed will be set to low." The "air conditioner" part may be replaced with the manufacturer's name, model name, part number, or a nickname set by the user for identification purposes.

[0105] In the example of the control screen 290 shown in Figure 9, the generation model MD2 determines that the instruction from the user (heating operation) corresponds to the air conditioner 100, which is a device capable of communication. The control screen 290 shown in Figure 9 displays the content of the control command (heating operation, set temperature 28°C, low airflow) transmitted to the air conditioner 100. The airflow may be the one set in the most recent operation, or it may be configured to use the one that is initially set. In this way, the air conditioner system 1 of this disclosure can receive instructions to the air conditioner 100 in the form of a conversation between the user and the generation model MD2 in the user terminal 200.

[0106] In the output area 292 of the control screen 290 shown in Figure 10, the text "Where are you going on your trip this year?" is displayed in a speech bubble as input received from the user. In response to this, the output area 292 of the control screen 290 in Figure 10 displays the text "This is not an instruction for the air conditioning, so we don't know. Would you like to do anything to operate the air conditioning?" which was generated by the command / notification generation unit 212 and output by the notification output unit 214.

[0107] Thus, in the air conditioning system 1 of this disclosure, instructions can be received for the air conditioner 100 in the form of a conversation between the user and the generative model MD2 in the user terminal 200. Moreover, as shown in Figure 10, if the input text from the user is inappropriate, the system does not force the user terminal 200 to perform incorrect control, nor does it remain unresponsive to the user's input text. Instead, it informs the user that control cannot be performed based on the input text and allows them to give correct instructions. After the result shown in Figure 10, the user re-enters appropriate text, and as shown in Figure 9, an appropriate control command is sent to the air conditioner 100.

[0108] In the output area 292 of the control screen 290 shown in Figure 11, the text "Turn on the air conditioner to the set temperature of 28°C for heating. Also humidify as it is dry." is displayed in a speech bubble as input received from the user. In response, the output area 292 of the control screen 290 in Figure 11 also shows text indicating that control commands with the following content were sent by the command / notification generation unit 212 and the command transmission unit 213: "The current room temperature is 22°C. A message has been sent to the air conditioner to start heating operation at the set temperature of 28°C. The current room humidity is 44%. The humidifier has been set to operate with a target humidity of 60% as the recommended humidity."

[0109] As shown in Figure 11, the air conditioning system 1 of this disclosure can receive instructions from the user regarding the operation of at least the air conditioner 100, and if the instructions are appropriate, it can also automatically generate and transmit control commands to surrounding communication-enabled devices that are tailored to the target device.

[0110] If, in response to the input text shown in Figure 11, there is no humidifier 157 that can receive instructions similar to the air conditioner 100, the command / notification generation unit 212 may generate a control command, generation model MD2, which states: "The current indoor temperature is 22°C. We have sent a message to the air conditioner to start heating operation at the set temperature of 28°C. The current indoor humidity is 44%. Humidification will be performed using the humidification function of the air conditioner."

[0111] In the output area 292 of the control screen 290 shown in Figure 12, the text "Turn on the air conditioner for cooling at a set temperature of 32°C." is displayed in a speech bubble as input received from the user. In response, the output area 292 of the control screen 290 in Figure 12 displays a notification of control failure, including the "reason," which is generated by the command / notification generation unit 212 and output by the notification output unit 214. The notification states, "According to your air conditioner's manual, the temperature that can be set for cooling operation is between 17°C and 30°C. What temperature would you like to set it to?" After this, the user specifies a temperature, and an appropriate control command is generated and sent to the air conditioner 100. If an inappropriate instruction like the one shown in Figure 12 is input by the user, the system will not either start heating operation at 32°C against the user's intention or ignore the user's instruction, but will instead allow the system to elicit an appropriate instruction from the user.

[0112] In the output area 292 of the control screen 290 shown in Figure 13, the text "Turn on the automatic filter cleaning function of the air conditioner." is displayed in a speech bubble as input received from the user. In response, the output area 292 of the control screen 290 in Figure 13 displays a notification of uncontrollable operation, including an "alternative," which was generated by the command / notification generation unit 212 and output by the notification output unit 214: "Your air conditioner does not have an 'automatic filter control function.' Checking the manual, it appears your air conditioner has an 'internal cleaning operation function.' Would you like to execute this?" After this, the user can add a message such as "Yes, please," which will then allow the appropriate command to be sent to the air conditioner 100.

[0113] The control screen 290 shown in Figure 13 illustrates an example where an alternative solution is presented because the content of the input statement for the air conditioner 100 installed in the target space does not correspond to the target air conditioner 100. In addition, the command / notification generation unit 212 may use the generation model MD2 to output a notification of uncontrollable status if the content of the input statement is an instruction for equipment not installed in the target space. The command / notification generation unit 212 can accept input statements indicating instructions for lighting equipment 154 and sound equipment 155, but if the specifications or number of those devices in the instruction do not match the equipment to be controlled, it may output a notification of uncontrollable status or present an alternative solution.

[0114] Figures 9-13 illustrate an example where a user, primarily an individual, controls a household air conditioner 100 via a user terminal 200. The air conditioner 100 and control program PG2 are not limited to household and individual users; they may also be for commercial use and shared by multiple people. Even in this case, a control command or a notification of uncontrollable operation is generated and transmitted or output in response to the input message from the person performing the operation on behalf of the others. In the case of commercial use where multiple air conditioners 100 are installed in the target space, the generation model MD2 may determine whether it is possible to output a control command in response to the input message based on construction information including the floor plan and installation location. For example, even if the construction information confirms that multiple air conditioners 100 are installed in a large space, if the input message is an instruction similar to a request for a relatively small space, such as "cool down to 24°C immediately," and it is determined that this is difficult even after referring to the horsepower included in the construction information, a notification of uncontrollable operation will be generated, and it may be possible to propose an alternative, such as operating the cooling system with the goal of reaching 25°C in 15 minutes.

[0115] In this way, the generation model MD2 is used to automatically output a command or a control failure notification depending on whether or not an appropriate command can be output from the user's instructions. This prevents the air conditioner 100 from receiving user instructions that are not compatible with the air conditioner 100, thereby preventing it from behaving against the user's wishes. Furthermore, since user instructions are not sent to the air conditioner 100 in an undecipherable state, the air conditioner 100 is not made to perform unnecessary processing, thereby improving the safety of equipment control and enhancing the user experience.

[0116] (Second Embodiment) In the second embodiment, the user terminal 200 executes processing using the generation model MD3 of the first server 310. The air conditioning system 1 in the second embodiment is the same as the air conditioning system 1 in the first embodiment, except for the processing procedure using the first server 310 which will be described below, so the same reference numerals are used for common components and detailed descriptions are omitted. In the second embodiment, the command / notification generation unit 212 function of the control unit 201 of the user terminal 200 is performed by the control unit 311 of the first server 310.

[0117] Figures 14 and 15 are flowcharts illustrating an example of the air conditioning process performed by the air conditioning system 1 in the second embodiment. For steps in Figures 14 and 15 that are common to the process shown in Figure 7 of the first embodiment, the same step numbers are used, and detailed explanations are omitted.

[0118] In the second embodiment, when the control unit 201 of the user terminal 200 receives an input message (S202), it transmits the received input message and the device information identified in step S201 to the first server 310 (step S213). In step S213, the control unit 201 transmits identification data of the user using the user terminal 200, or data indicating the location, along with the device information and the input message.

[0119] In the second embodiment, the first server 310 receives an input message and equipment information from the user terminal 200 (step S311). Based on the control program PG3, the control unit 311 of the first server 310 requests relevant information from the second server 320 based on the transmitted input message and equipment information (step S312). In step S312, the control unit 311 may add data such as the model, product name, model number, and specifications of the air conditioning equipment to the request as equipment information. In step S312, the control unit 311 may add user identification data or data indicating location to the request.

[0120] The second server 320 receives a request for relevant information (step S321), and the control unit 321 identifies knowledge data such as user information, manuals, driving data, and weather information from the database DB based on the information included in the request (step S322). The control unit 321 either reads the identified knowledge data or obtains the reference location of the knowledge data (step S323) and sends it to the first server 310 (step S324).

[0121] The first server 310 receives knowledge data or a reference destination (step S313). The control unit 311 specifies the received knowledge data itself or the reference destination, and generates a prompt to be given to the generation model MD3 along with the input statement received in step S311 and information about the controlled device (step S314). In step S314, the control unit 311 may generate the prompt by reading one stored in the storage unit 312, automatically generate it based on a template, or generate it using the generation model MD3.

[0122] The prompt generated in step S314, similar to the first embodiment, instructs the system to generate a control command to the air conditioner 100 corresponding to the input statement if the given input statement corresponds to the control of the air conditioner 100, and to generate a control failure notification if the input statement does not correspond to the control of the air conditioner 100.

[0123] The prompt generated in step S314 may include a description that outputs a control command or a control failure notification based on the recommended heating / cooling settings, recommended temperature settings, target humidity, etc., based on the outdoor temperature, humidity, or weather information of the space where the controlled air conditioner 100 is installed, depending on the location of the user terminal 200. The prompt may also include a description that instructs the output of an appropriate control command or a control failure notification depending on the time of year, such as disabling heating operation in the summer.

[0124] The prompt generated in step S314 may be given instructions to include the reason, or instructions to include the addition of alternative information. Otherwise, the prompt generation process may be the same as the prompt generation (S203) shown in Figure 7 of the first embodiment.

[0125] The control unit 311 provides the input text and device information received in step S311, the knowledge data itself or the specified reference destination received in step S313, and the prompt generated in step S14 to the generation model MD3 (step S315). In step S315, if there is a conversation history, such as a continuing conversation, the control unit 311 may also provide this to the generation model MD3. If the generation model MD3 continues to accept input text from the user of the user terminal 200, it retains past user input and may output a control command or a control failure notification reflecting the past data.

[0126] The control unit 311 receives a control command or a control failure notification output from the generated model MD3 (step S316), and transmits the received control command or control failure notification to the user terminal 200 (step S317).

[0127] The control unit 201 of the user terminal 200 receives a control command or a notification that control is not possible (step S215). The control unit 201 determines whether or not a control command has been received (step S216), and if it determines that a control command has been received (S216: YES), it transmits the control command to each device, including the corresponding air conditioner 100, using the function of the command transmission unit 213 (S207). The control unit 201 displays a screen on the display unit 209 indicating that a control command has been transmitted (S208), and terminates the process.

[0128] If the control unit 201 determines in step S216 that it has not received a control command (S216: NO), it has received a notification that control is impossible, and therefore, using the function of the notification output unit 214, it displays the notification on the display unit 209 (S209).

[0129] In the second embodiment, the first server 310 can perform processing using the knowledge data obtained from the second server 320. Since manuals for various models can be obtained as knowledge data, more accurate control commands can be output from the user terminal 200 when targeting a variety of devices compared to when the user terminal 200 obtains the data. Furthermore, by extracting and using weather information and time information corresponding to the user's location from the second server 320, even more appropriate control commands or notifications of uncontrollable devices can be output from the user terminal 200.

[0130] (Third Embodiment) In the third embodiment, the user terminal 200 keeps a history of user input sentences to the user generation model MD2 and the output obtained from the generation model MD2, and performs a function to detect user abnormalities based on the differences in the history of user input sentences. The air conditioner system 1 of the third embodiment is the same as the air conditioner system 1 of the first embodiment, except for the details of the processing described below, so the same reference numerals are used for common components and detailed descriptions are omitted.

[0131] Figure 16 is a functional block diagram showing the functions of the user terminal 200 in the air conditioning system 1 of the third embodiment. In the third embodiment, the control unit 201 of the user terminal 200 functions as an instruction receiving unit 211, a command / notification generation unit 212, a command transmission unit 213, a notification output unit 214, and a state prediction unit 215, based on the control program PG2.

[0132] The control unit 201, which functions as a state estimation unit 215, uses the generation model MD2 to estimate the user's state from newly received input statements based on the history of user instructions to the air conditioner 100 stored in the storage unit 202 or an external storage device. The state estimation unit 215 may also estimate the state without using the generation model MD2.

[0133] Figures 17 and 18 are flowcharts showing an example of the air conditioning control procedure by the air conditioning system 1 in the third embodiment. Of the processing steps shown in Figures 17 and 18, those steps that are common to the processing steps shown in Figure 7 of the first embodiment are given the same step numbers and detailed explanations are omitted. The following processing steps begin when the user starts the control program PG2 on the user terminal 200.

[0134] In the third embodiment, after receiving an input statement (S202), the control unit 201 reads the history of input statements and control commands or notifications stored in a process described later (step S222) and generates a prompt (S203). In step S203, for example, the control unit 201 uses the functions of the command / notification generation unit 212 to refer to the read history and generates a prompt that instructs the generation of an appropriate control command or a notification of uncontrollable state based on past trends.

[0135] When the control unit 201 receives a control command or a notification of uncontrollable state from the generation model MD2 (S205), it stores the received control command or notification of uncontrollable state, along with the input statement received in step S202, as a history of the interaction between the user and the generation model MD2 (step S225).

[0136] In the third embodiment, if the control unit 201 determines in step S206 that the output from the generation model MD2 is a control command (S206: YES), it uses the function of the state estimation unit 215 to estimate the user's state based on the difference between the control command acquired in step S205 and the control command in the history read in step S222 (step S227). In step S227, the control unit 201 provides the generation model MD2 with the control command acquired in step S205, the history read in step S222, and a prompt instructing it to estimate the state based on the difference with the history, and estimates the state. The control unit 201 may also add an instruction to the prompt generated in step S203 to estimate the state based on a comparison of the output control command with the history of past control commands. In step S227, the control unit 201 may identify the difference by statistical processing, such as the difference in set temperature, and estimate the state from a comparison of an index value that evaluates the magnitude of the difference with a predetermined value.

[0137] In step S227, the control unit 201 determines whether or not an abnormal state has been inferred (step S228). If the control unit 201 determines that an abnormal state has been inferred (S228: YES), it stores the stopped state in the storage unit 202 in association with the user identification data of the user using the user terminal 200, in order to stop accepting subsequent input messages (step S229). The control unit 201 detects an abnormal state and outputs a message to the display unit 209 indicating that it will not accept subsequent inputs (step S230), and terminates the process. In step S230, the control unit 201 may also request the first server 310 to notify the family, emergency contacts, and other notification recipients set for the user of the abnormal state.

[0138] If it is determined in step S227 that no abnormal condition is suspected (S228: NO), commands are sent to each device (S207), and a screen indicating that control commands have been sent is displayed on the display unit 209 (S208), and the process ends.

[0139] If it is determined in step S206 that it is not a control command (S206: NO), the generation model MD2 is instructed to generate a speech statement related to the received input sentence that extracts the missing information necessary to output a control command for the air conditioner 100 (step S231). The speech statement in step S231 may have instructions added to the prompt in step S203 so that it can be acquired in step S206 as a notification that control is not possible. The control unit 201 displays the speech statement generated in step S231 on the display unit 209 (step S232), stores the generated speech statement in the storage unit 202 as history (step S233), and returns the process to step S202.

[0140] Figure 19 shows an example of notification of an abnormal state on the control screen 290. Figure 19 is displayed when the user starts the control program PG2 for sending control commands to the air conditioner 100 from the user terminal 200, similar to the control screen 290 shown in Figures 9-13 in the first embodiment. In the example in Figure 19, the text "Turn on the air conditioner for cooling at a set temperature of 22°C." is displayed in a speech bubble as input received from the user. In response to this, the output area 292 of the control screen 290 in Figure 19 is inferred by the function of the state inference unit 215 from the history, based on situations such as the cooling operation never being set to below 25°C, or the instruction to turn on cooling at 22°C despite the season or outside temperature being winter and the room temperature being below 20°C. As shown in Figure 19, an abnormal condition may only be inferred when, rather than from a single input message, the command / notification generation unit 212 repeatedly issues a notification that control is impossible, such as "The room temperature is below 20°C. Cooling operation is not possible," yet the instruction to turn on the air conditioner is repeatedly issued.

[0141] Figure 20 shows an example of how spoken text is displayed on the control screen 290. Similar to Figure 19, Figure 20 is displayed when the user starts the control program PG2 on the user terminal 200 to send control commands to the air conditioner 100. In the example in Figure 20, the ambiguous text "Please warm it up moderately." is displayed in a speech bubble as the input text received from the user. In response, the command / notification generation unit 212 of the user terminal 200 generates a question for the air conditioner 100, stating that the instruction is ambiguous and a control command cannot be generated, and requests the necessary information to generate a control command based on the word "warm." On the control screen 290 in Figure 20, the following spoken text is generated and displayed: "Starting heating operation. The recommended setting temperature is 23°C, is that OK?" This assumes that the input text "Please warm it up" is for heating operation and asks for the necessary setting information. Furthermore, if the user only provides the instruction "That's fine," and information such as airflow volume or direction is missing, the control screen 290 in Figure 20 generates a speech statement to obtain further necessary information, such as "We recommend a low airflow volume, is that alright?"

[0142] Thus, in the third embodiment of the air conditioning system 1, the user's input sentences and the output from the generation model MD2 or generation model MD3 are recorded as a conversation history, thereby enabling conversational communication to infer abnormalities from new input sentences that are not in the history or to acquire information. In the third embodiment as well, it is possible to prevent sending user instructions to the air conditioner 100 even if they do not correspond to the air conditioner 100, thereby preventing the system from inducing behavior contrary to the user's intentions. User instructions are not sent to the air conditioner 100 in an undecipherable state, so the air conditioner 100 is not made to perform unnecessary processing, thereby improving the safety of equipment control and the user experience.

[0143] In the third embodiment of the air conditioning system 1, it was explained that the user terminal 200 generates control commands or notifications of uncontrollable state using generation model MD2. However, in the third embodiment as well, the generation may be performed using generation model MD3 of the first server 310.

[0144] The embodiments disclosed above are illustrative in all respects and not restrictive. The scope of the invention is indicated by the claims, and all modifications within the meaning and scope of the claims are equivalent.

[0145] 1. Air conditioning system 100. Air conditioner 151. Air purifier 152. Circulator 153. Diffuser 154. Lighting equipment 155. Audio equipment 156. Outdoor air handling unit 157. Humidifier 158. Dehumidifier 200. User terminal 201. Control unit 202. Memory unit 203. Communication unit 208. Operation unit 209. Display unit PG1, PG2, PG3. Control programs MD1, MD2, MD3. Generation models

Claims

1. An air conditioner control method that receives an input statement relating to an air conditioner, generates a control command to the air conditioner corresponding to the input statement if the input statement corresponds to the control of the air conditioner, generates a control failure notification corresponding to the input statement if the input statement does not correspond to the control of the air conditioner, uses a generation model to generate such a command, and causes a computer to execute a process of transmitting the generated control command to the air conditioner or outputting the generated notification.

2. The air conditioner control method according to claim 1, which causes the computer to perform a process to generate the notification such that the input statement includes the reason why it does not correspond to the control of the air conditioner.

3. The air conditioner control method according to claim 1, which causes the computer to perform a process of adding information about an alternative to the input statement to the notification of uncontrollable state and outputting it.

4. The air conditioner control method according to claim 1, which stores the received input sentence and the generated control command or control failure notification as a history, and causes the computer to execute a process to generate the control command or control failure notification using the generation model and the history in response to a new input sentence.

5. The air conditioner control method according to claim 4, which causes the computer to perform a process to infer the state of the user who provided the input statement, based on the difference between the control command corresponding to the new input statement and the control command for the air conditioner in the history.

6. The air conditioner control method according to claim 5, wherein if an abnormal state is inferred from the user's state, the computer is instructed to execute a process to stop accepting input messages from the user.

7. The air conditioner control method according to any one of claims 1 to 4, wherein if the input sentence does not correspond to the control of the air conditioner, the computer is instructed to perform a process to generate an utterance related to the input sentence.

8. The air conditioner control method according to any one of claims 1 to 4, wherein the input statement includes information relating to a lighting device or sound equipment in addition to the air conditioner.

9. An air conditioner control method according to any one of claims 1 to 4, wherein information on the temperature, humidity, or weather inside and outside the space in which the air conditioner is installed is provided to the generation model, and the computer is instructed to perform a process of generating a control command to the air conditioner or a notification of control failure using the generation model.

10. An air conditioner control method according to any one of claims 1 to 4, which causes the computer to perform a process of providing the generation model with a prompt instructing it to generate a control command to the air conditioner corresponding to the input statement if the input statement corresponds to the control of the air conditioner, and to generate a notification of uncontrollable state corresponding to the input statement if the input statement does not correspond to the control of the air conditioner, and the input statement.

11. The air conditioner control method according to claim 10, which causes the computer to perform a process of providing the generation model with a document including an instruction manual for the air conditioner, along with the prompt and the input statement.

12. A computer program that causes a computer communicating with an air conditioner to receive an input statement related to the air conditioner, generate a control command to the air conditioner corresponding to the input statement if the input statement corresponds to the control of the air conditioner, or generate a notification of control failure corresponding to the input statement if the input statement does not correspond to the control of the air conditioner, using a generation model, and then transmit the generated control command to the air conditioner or output the generated notification.

13. An air conditioner control device comprising a processing unit that receives an input statement relating to an air conditioner, generates a control command to the air conditioner corresponding to the input statement if the input statement corresponds to the control of the air conditioner, or generates a notification of control failure corresponding to the input statement if the input statement does not correspond to the control of the air conditioner, using a generation model, and then transmits the generated control command to the air conditioner or outputs the generated notification.

Images