Information processing system, information processing method, and information processing program

Abstract

An information processing system (10) comprises at least one processor. Said at least one processor: inputs, to an LLM (25A) (language model), a prompt indicating an instruction to generate a control parameter for an environment adjustment facility (F) provided in a target space (TS); and controls the environment adjustment facility (F) on the basis of the control parameter generated by the LLM (25A).

Description

Information Processing System, Information Processing Method, and Information Processing Program

[0001] Embodiments of the present invention relate to an information processing system, an information processing method, and an information processing program.

[0002] A system for controlling environmental control equipment such as air conditioning equipment arranged in a specific target space such as a building or a square is known. For example, a system for controlling environmental control equipment according to the environmental state such as the temperature of the target space acquired by a sensor is known. In addition, control of environmental control equipment is performed according to a prediction result obtained by predicting the environmental state of the target space at a future timing using past temperature data and weather forecast data.

[0003] However, in the prior art, as the reference data such as past temperature data and weather forecast data increases, analysis and analysis become more complicated, and it may be difficult to perform equipment control for easily realizing the required environment with a simple configuration.

[0004] Japanese Patent No. 7104357

[0005] The present invention has been made in view of the above, and an object thereof is to provide an information processing system, an information processing method, and an information processing program that can perform equipment control for easily realizing a required environment with a simple configuration.

[0006] The information processing system according to the embodiment includes at least one processor. The at least one processor inputs a prompt representing an instruction to generate control parameters of environmental control equipment provided in a target space to a language model, and controls the environmental control equipment based on the control parameters generated by the language model.

[0007] FIG. 1 is an explanatory diagram of an example of the system according to the embodiment. FIG. 2 is a schematic diagram of an example of a display screen. FIG. 3 is an explanatory diagram of an example of processing by a prompt generation unit. FIG. 4 is a sequence diagram showing an example of the flow of information processing executed by the system. FIG. 5 is a flowchart showing an example of the flow of information processing executed by the information processing system according to the embodiment. FIG. 6 is a hardware configuration diagram of an example of the information processing system and an external system.

[0008] The embodiments of the information processing system, information processing method, and information processing program will be described in detail below with reference to the attached drawings.

[0009] Figure 1 is an explanatory diagram of an example of System 1 of this embodiment.

[0010] System 1 is a system for adjusting the target space TS to the desired environment by controlling the environmental adjustment equipment F installed in the target space TS.

[0011] The target space TS is the real space that is subject to environmental adjustment by System 1. The target space TS can be any predetermined space in the real world. For example, the target space TS may be a space inside a specific building, a predetermined space outdoors, etc. In this embodiment, the form in which the target space TS is a space inside a predetermined building will be described as an example.

[0012] Environmental adjustment equipment F is equipment installed in the target space TS that can adjust the environment of the target space TS. In other words, environmental adjustment equipment F is equipment that can change the sensory stimuli of a user (person) by adjusting the environment of the target space TS. Environmental adjustment equipment F includes, but is not limited to, air conditioning equipment, lighting fixtures, ventilation equipment, humidifiers, fans, water heaters, curtain opening and closing devices, electric aroma pots, etc.

[0013] The target space TS is equipped with one or more types of environmental control equipment F. Furthermore, the target space TS may be equipped with multiple units of the same type of environmental control equipment F. In this embodiment, a configuration in which multiple types of environmental control equipment F are provided in the target space TS, and at least one type of environmental control equipment F is provided in multiple units, will be described as an example.

[0014] The target space TS is equipped with an imaging device C, a microphone M, and an environmental sensor S. Furthermore, users present in the target space TS are wearing wearable sensors W.

[0015] The imaging device C acquires image data of the target space TS by imaging. In this embodiment, the image data will be referred to as the image.

[0016] Microphone M collects sound within the target space TS and acquires audio data. In this embodiment, audio data will be referred to as "sound."

[0017] The environmental sensor S is a sensor that acquires detection information representing the detection result of the environmental state of the target space TS. The detection information can be any information that represents the detection result of the environmental state of the target space TS. Environmental state includes, but is not limited to, temperature, humidity, concentration of specific chemical substances, odor, illuminance, color temperature, airflow, wind speed, atmospheric pressure, etc. Also, the detection information detected by the environmental sensor includes, but is not limited to, information that represents, such as temperature, humidity, concentration of specific chemical substances, odor, illuminance, color temperature, airflow, wind speed, atmospheric pressure, etc. Specific chemical substances include, but is not limited to, carbon dioxide, nitrogen compounds, etc. The wearable sensor W acquires vital data such as the user's heart rate, skin temperature, and amount of sweat.

[0018] System 1 comprises an information processing system 10, an external system 20, a shooting device C, a microphone M, an environmental adjustment device F, and an environmental sensor S.

[0019] The information processing system 10, the external system 20, the imaging device C, the microphone M, the environmental sensor S, the wearable sensor W, and the environmental adjustment equipment F are connected wirelessly or via wired communication through a network NW or the like.

[0020] The external system 20 is one or more information processing devices located outside the information processing system 10. The external system 20 is composed of one or more dedicated or general-purpose computers.

[0021] The external system 20 includes a communication unit 21, an input unit 22, an output unit 23, a storage unit 24, and a control unit 25. The communication unit 21, input unit 22, output unit 23, storage unit 24, and control unit 25 are connected to each other via a bus or the like.

[0022] The communication unit 11 communicates with the information processing system 10, etc., via a network NW, etc. The input unit 12 accepts various operations from the user. The output unit 23 outputs various types of information. The output unit 23 includes a display for showing various types of information, a speaker for outputting various types of sounds, etc.

[0023] The storage unit 24 stores various types of data. The storage unit 24 may be, for example, a semiconductor memory element such as RAM (Random Access Memory) or flash memory, a hard disk, or an optical disk. The storage unit 27 may be a storage device located outside the external system 20. The storage unit 27 may also be a storage medium that stores or temporarily stores programs and various types of information downloaded via a LAN (Local Area Network) or the Internet.

[0024] The control unit 25 performs information processing in the external system 20. The control unit 25 is implemented by one or more processors. For example, the control unit 25 may be implemented by having a processor such as a CPU (Central Processing Unit) execute a program, i.e., by software.

[0025] The control unit 25 includes a language model. The language model may be implemented by a dedicated IC (Integrated Circuit) or other processor, i.e., hardware. The language model may also be implemented using a combination of software and hardware. Furthermore, the language model may be provided in the information processing system 10, which will be described later.

[0026] Language models are generative AI (Artificial Intelligence) computer programs and systems used in the field of natural language processing (NLP) that are capable of tasks such as question answering and text generation. Examples of language models include large language models (LLMs), generative language models (e.g., Generative Pretrained Transformers (GPTs)), representational language models (e.g., Bidirectional Encoder Representations from Transformers (BERTs)), and / or any other type of language model.

[0027] In this embodiment, an example will be described in which the external system 20 includes LLM25A as a language model.

[0028] Next, the information processing system 10 will be described.

[0029] The information processing system 10 is one or more information processing devices. The information processing system 10 is composed of one or more dedicated or general-purpose computers.

[0030] The information processing system 10 comprises a communication unit 11, an input unit 12, an output unit 13, a storage unit 14, and a control unit 15. The communication unit 11, input unit 12, output unit 13, and storage unit 14 and the control unit 15 are communicated to each other via a bus or the like. At least one of the external system 20, the output unit 13, and the storage unit 14 functions as an output device 17, which is the output destination for various types of information by the control unit 15. The information processing system 10 may further be configured to include at least one of the above-mentioned imaging device C, microphone M, environmental sensor S, and environmental adjustment equipment F. These imaging device C, microphone M, environmental sensor S, and environmental adjustment equipment F are provided in the target space TS and should be communicated to the control unit 15 of the information processing system 10 via a network NW or the like.

[0031] The communication unit 11 communicates with an external system 20, etc., via a network NW, etc. The input unit 12 accepts various operations from the user. The output unit 13 outputs various information. The output unit 13 is a display that shows various information, a speaker that outputs various sounds, etc. At least one of the input unit 12 and the output unit 13 may be configured to be located outside the information processing system 10 and to be connected to the control unit 15 in a manner that allows communication.

[0032] The storage unit 14 stores various types of data. The storage unit 14 can be, for example, a semiconductor memory element such as RAM or flash memory, a hard disk, or an optical disc. The storage unit 14 may also be a storage device located outside the information processing system 10. Alternatively, at least some of the information stored in the storage unit 14 may be stored in a storage device located outside the information processing system 10. Furthermore, the storage unit 14 may be a storage medium on which programs and various types of information are downloaded and stored or temporarily stored via a LAN (Local Area Network) or the Internet.

[0033] The control unit 15 performs information processing in the information processing system 10.

[0034] The control unit 15 includes a first prompt generation unit 15A, a second prompt generation unit 15B, a third prompt generation unit 15C, a prompt generation unit 15D, an equipment control unit 15E, and an output control unit 15F.

[0035] The first prompt generation unit 15A, the second prompt generation unit 15B, the third prompt generation unit 15C, the prompt generation unit 15D, the equipment control unit 15E, and the output control unit 15F are implemented by one or more processors. For example, each of the above units may be implemented by having a processor such as a CPU execute a program, i.e., by software. Each of the above units may be implemented by a dedicated IC or other processor, i.e., by hardware. Each of the above units may be implemented by using both software and hardware. When multiple processors are used, each processor may implement one of the above units, or two or more of the above units.

[0036] Alternatively, at least one of the above-mentioned components included in the control unit 15 may be mounted on an external information processing device such as an external system 20 that is connected to the information processing system 10 via a network NW or the like.

[0037] The first prompt generation unit 15A generates a first prompt. The first prompt is a prompt that represents the content of the first adjustment request made by the user to the environmental adjustment equipment F.

[0038] A prompt is an instruction or input sentence given to an AI model.

[0039] The first adjustment request is the adjustment request entered by the user. In other words, the first adjustment request is the adjustment request desired by the user. The adjustment request is information that represents the request for adjustment of the environmental conditions for the environmental adjustment equipment F.

[0040] The first prompt generation unit 15A acquires the content of the first adjustment request input by the user through the operation instructions of the input unit 12. For example, the output control unit 15F displays the display screen on the output unit 13. The output control unit 15F outputs various information to the output device 17. The first prompt generation unit 15A then acquires the content of the first adjustment request received via the display screen.

[0041] Figure 2 is a schematic diagram of an example of the display screen 30.

[0042] The display screen 30 is an example of a display screen that the output control unit 15F outputs to the output device 17. The output control unit 15F may also display the display screen 30 on the output unit of each information processing terminal operated by each of multiple users. The output unit is the same as the output unit 13. An information processing terminal is an information processing device operated by a user. An information processing terminal is, for example, a smartphone, a mobile terminal, a tablet terminal, a personal computer, etc., but is not limited to these. An information processing terminal only needs to be connected to the information processing system 10 in a communicative manner via a network NW, etc. In this embodiment, a configuration in which the output control unit 15F outputs the display screen 30 to the output unit 13 will be described as an example.

[0043] The display screen 30 includes an input screen area 32 and a control information display area 34.

[0044] The input screen area 32 is an area including an input field for receiving an input by the user. The control information display area 34 is a display area for control information described later. Details of the control information display area 34 will be described later.

[0045] The input screen area 32 includes a first prompt input area 32A, a second prompt input area 32B, and a third prompt input area 32C. Details of the second prompt input area 32B and the third prompt input area 32C will be described later.

[0046] The first prompt input area 32A is a display area for receiving an input of the content of the first adjustment request by the user. In the first prompt input area 32A, for example, there are a statement prompting the input of the content of the first adjustment request and an input field for the content of the first adjustment request. FIG. 2 shows an example where the statement “Please input the required content for the environmental adjustment equipment” is displayed as the statement prompting the input of the content of the first adjustment request. The user inputs the content of the first adjustment request into the input field provided below the display position of the statement by operating the input unit 12 or the like.

[0047] For example, the user inputs information regarding a desire for realizing a required environment in which the user present in the target space TS is predicted to feel comfortable, such as the state of the user currently present in the target space TS, the predicted required environment of the target space TS that the user present in the target space TS is predicted to require, the physical condition of the user present in the target space TS, the classification of the users present in the target space TS, etc., as the content of the first adjustment request.

[0048] The state of the user is during lunch break, during break time, busy, etc. The predicted required environment is, for example, a relaxing atmosphere, an environment where concentrated thinking is possible, etc. The physical condition of the target space TS is, for example, in a bad condition, in good condition, etc. The classification of the users present in the target space TS is the ratio of men and women, etc.

[0049] Figure 2 shows an example where the first adjustment request received via the first prompt input area 32A was "It's a summer workplace. It's currently lunchtime, please create a relaxing lighting environment. Also, please create a temperature environment that takes into account the ratio of men to women."

[0050] The first prompt generation unit 15A receives the first adjustment request content entered in the input field from the input unit 12 and the output unit 13, thereby acquiring the content of the first adjustment request from the user to the environmental adjustment equipment F.

[0051] The first prompt generation unit 15A then generates a first prompt corresponding to the acquired first adjustment request content.

[0052] In detail, the first prompt generation unit 15A generates a prompt representing an instruction to generate control parameters for the environmental adjustment equipment F as the first prompt, in order to realize the target space TS of the environmental state represented by the first adjustment request content.

[0053] The control parameters of the environmental control equipment F are parameters used to control the environmental control equipment F. In detail, the control parameters include information representing the set values ​​of the environmental conditions of the environmental control equipment F, the operating mode of the environmental control equipment F, etc. As mentioned above, environmental conditions include, for example, temperature, humidity, concentration of specific chemical substances, odor, illuminance of lighting, color temperature of lighting, airflow rate, wind speed, atmospheric pressure, etc. Therefore, the control parameters include information representing the set values ​​for controlling at least one of these multiple types of environmental conditions by the environmental control equipment F. The operating mode is information representing the cycle of operation and stopping, the components to be operated on (specific air outlets, etc.), etc.

[0054] The first prompt generation unit 15A may generate the first prompt in a known manner based on the content of the first adjustment request.

[0055] For example, as shown in Figure 2, let's assume that the first adjustment request received is "It's a summer workplace. It's currently lunchtime, please create a relaxing lighting environment. Also, please create a temperature environment that takes into account the ratio of men to women." In this case, the first prompt generation unit 15A generates a first prompt that includes the first adjustment request and an instruction to generate control parameters for the environmental adjustment equipment F to realize the environmental state represented by the first adjustment request.

[0056] The first prompt generation unit 15A may also receive the first adjustment request content from each of the information processing terminals of multiple users. In this case, the first prompt generation unit 15A analyzes the multiple first adjustment request content received and identifies one first adjustment request content for the user with the worst physical condition using a known method such as generation AI. Then, the first prompt generation unit 15A can generate a first prompt based on the identified single first adjustment request content.

[0057] Furthermore, the first prompt generation unit 15A may generate a first prompt that includes a plurality of first adjustment request contents obtained from the information processing terminals of multiple users, a target space TS representing the environmental state expressed by each of the plurality of first adjustment request contents, and an instruction to generate control parameters for the environmental adjustment equipment F that will be most comfortable for the user who is feeling the worst.

[0058] Returning to Figure 1, we continue the explanation.

[0059] The second prompt generation unit 15B generates a second prompt. The second prompt is a prompt that represents the content of the second adjustment request.

[0060] The second adjustment request is an adjustment request that corresponds to the analysis result of at least one of the captured images, audio, and vital data from the wearable sensor W of the target space TS.

[0061] In this embodiment, the second prompt generation unit 15B generates a second prompt representing a second adjustment request content corresponding to the analysis result obtained by analyzing at least one of the captured image, audio, and vital data, based on the analysis request content received as input.

[0062] The analysis request concerns the analysis of at least one of the following: captured images, audio, and vital data. Specifically, the analysis request concerns information about users (people) included in the target space TS. More specifically, the analysis request describes at least one of the user attributes and user status included in the target space TS. User attributes include the gender of users included in the target space TS, the ratio of males and females, age, age group ratio, etc. User status includes the degree of drowsiness, degree of physical condition (good or bad), clothing, comfort level, discomfort level, alertness level, etc., obtained through facial expression analysis, audio analysis, vital data analysis, etc. User status may also be obtained by audio analysis of conversations between multiple users. User status may also be obtained by the amount of clothing worn, obtained through image analysis.

[0063] The analysis results are obtained by analyzing at least one of the captured image audio and vital data based on the analysis request, and represent at least one of the user's attributes and / or state present in the target space TS.

[0064] The second prompt generation unit 15B acquires the analysis request content entered by the user through the operation instructions of the input unit 12. For example, the output control unit 15F displays the display screen 30 on the output unit 13. The second prompt generation unit 15B then acquires the analysis request content received via the display screen 30.

[0065] Let's explain using Figure 2. For example, the output control unit 15F outputs the display screen 30 shown in Figure 2 to the output device 17.

[0066] The second prompt generation unit 15B receives the analysis request details and the selection of at least one of the captured image and audio files to be analyzed, via the second prompt input area 32B on the display screen 30.

[0067] The second prompt input area 32B is a display area for receiving input for selecting at least one of the captured image and audio files to be analyzed, and for inputting the content of the analysis request for the target of analysis. The second prompt input area 32B includes, for example, text prompting the user to select a file, a display field for the selected file, text prompting the user to input the content of the analysis request, and an input field for the content of the analysis request. Figure 2 shows an example in which "Select File" is displayed as the text prompting the user to select a file, and "Please enter the content of the analysis request for the file to be used for adjusting the environmental adjustment equipment" is displayed as the text prompting the user to input the content of the analysis request. The user operates the input unit 12, etc., to select at least one of the desired captured image and audio files and to input the content of the analysis request in the input field provided below the display position of the text.

[0068] Figure 2 shows an example where the user selects images of the target space (TS) as the files to be analyzed. Figure 2 also shows an example where the user inputs "number of people and gender ratio in the target space" as the analysis request details.

[0069] The second prompt generation unit 15B receives at least one of the selected captured video and audio, and the input analysis request content from the input unit 12 and output unit 13, thereby acquiring at least one of the captured image and audio to be analyzed, as well as the analysis request content.

[0070] The second prompt generation unit 15B then generates a second prompt representing a second adjustment request content corresponding to the analysis result obtained by analyzing the target of analysis, based on at least one of the captured image and audio of the target of analysis and the analysis request content received as input.

[0071] In detail, the second prompt generation unit 15B obtains analysis results by analyzing the target of analysis, which is at least one of the acquired captured image and audio, using known image processing techniques, audio analysis techniques, etc., in accordance with the analysis request content received as input. Known methods may include, for example, generation AI. For example, as shown in Figure 2, consider a scenario where a captured image of the target space TS is selected as the target of analysis, and the analysis request content is "number of people present in the target space and the male-female ratio." In this case, the second prompt generation unit 15B analyzes the captured image, which is the target of analysis, using known image analysis methods, etc., to derive the number of people present in the target space TS and the male-female ratio included in the target space TS as analysis results.

[0072] The second prompt generation unit 15B then generates a second prompt that represents the derived analysis result and an instruction to generate control parameters for the environmental adjustment equipment F, which will realize an environmental state in the target space TS that users in the target space TS will perceive as comfortable, in accordance with the analysis result. The second prompt generation unit 15B can generate the second prompt, which includes the analysis result and the generation instruction, using a known method.

[0073] Returning to Figure 1, we continue the explanation.

[0074] The third prompt generation unit 15C generates a third prompt. The third prompt is a prompt that includes environmental element information relating to the environmental elements of the target space TS.

[0075] Environmental elements are factors that can influence the environmental state of the target space TS.

[0076] Environmental element information refers to information that represents environmental elements. In detail, environmental element information includes at least one of the following: detection information, equipment information, building information, and social information.

[0077] As described above, detection information refers to information that represents the detection results of the environmental state of the target space TS by the environmental sensor S. The detection information included in the environmental element information may be any information that represents one or more types of current and past detection results detected by the environmental sensor S in the target space TS. As described above, the detection information may include, for example, temperature, humidity, concentration of a specific chemical substance, odor, illuminance of lighting, color temperature of lighting, airflow, wind speed, atmospheric pressure, etc.

[0078] Equipment information refers to information relating to the environmental control equipment F. More specifically, the equipment information includes information relating to at least one of the specifications and installation location of the environmental control equipment F. The specifications information represents the specifications of the environmental control equipment F. Specifically, the specifications information represents, for example, the range of environmental elements that can be adjusted by the environmental control equipment F, such as temperature and brightness, the characteristics of the environmental control equipment F, the location and number of air outlets, etc. The characteristics of the environmental control equipment F include, for example, the relationship between the range of environmental elements that can be adjusted by the environmental control equipment F and the electricity charges. The installation location represents the location where the environmental control equipment F is installed in the target space TS. Furthermore, the equipment information may also include information representing the type and number of environmental control equipment F installed in the target space TS.

[0079] Building information refers to information about buildings included in the target space TS. Buildings included in the target space TS refer to the buildings that constitute the target space TS, and various buildings placed within the target space TS. Building information includes, for example, information representing the type of building that constitutes the target space TS, the material of the building, the age of the building, the year of construction, the building's purpose, and the number, location, and size of structures (windows, etc.) that may affect the environmental conditions of the target space TS. The type of building is information representing the type of building that constitutes the target space TS, such as concrete, reinforced concrete, or wood. The material of the building is information representing the material of the components such as walls and floors that constitute the target space TS. In other words, building information includes information on at least one of the following: the number of windows, the arrangement of windows, the material, and the age of the building, which are included in buildings that constitute the target space TS or are installed in the target space TS.

[0080] Social information refers to social information relating to the target space TS. More specifically, social information includes at least one of the following: preference trend information relating to users' preferences regarding the environment in the real space including the target space TS; weather information relating to the weather in the real space including the target space TS; age group information of users utilizing the target space TS; employment status information of users utilizing the target space TS; and date and time information.

[0081] User preferences regarding the environment in the real space including the target space TS include, for example, this year's clothing trends, clothing trends by gender and age group, etc. Weather information for the real space including the target space TS includes, for example, information representing the weather, temperature, humidity, etc. of the real space. Age group information includes, for example, information representing the ratio of the number of users belonging to each age group using the target space TS. Employment status information includes, for example, lunch break times, busy periods, etc.

[0082] In this embodiment, the third prompt generation unit 15C acquires at least a portion of the environmental element information by inputting it from the user. Alternatively, the third prompt generation unit 15C may acquire at least a portion of the environmental element information from a predetermined data source.

[0083] The third prompt generation unit 15C then generates a third prompt that includes the acquired environmental element information.

[0084] For example, the third prompt generation unit 15C acquires environmental element information input by user operation instructions on the input unit 12. For example, the output control unit 15F displays the display screen 30 on the output unit 13. The third prompt generation unit 15C acquires environmental element information received via the display screen 30.

[0085] Let's explain using Figure 2. For example, the output control unit 15F outputs the display screen 30 shown in Figure 2 to the output device 17.

[0086] The third prompt generation unit 15C receives environmental element information input via the input area 32C for the third prompt on the display screen 30.

[0087] The third prompt input area 32C is a display area for receiving input of environmental element information. The third prompt input area 32C includes, for example, text prompting the input of environmental element information and input fields for environmental element information. Figure 2 shows an example that includes text prompting the input of equipment information, which is an example of environmental element information, input fields for equipment information for each type of environmental adjustment equipment F provided in the target space TS, and an add button for additional equipment information input items by the user.

[0088] Figure 2 shows an example where the message "Please enter equipment information" is displayed to prompt the user to enter equipment information, and the words representing "air conditioning," "lighting," and "air purifier" are displayed as environmental control equipment F. The user enters the equipment information into the input field provided near the display location of the words by operating the input unit 12, etc.

[0089] Figure 2 shows the equipment information for the environmental control equipment F "Air Conditioning" as follows: "Number of air outlets: 3, Maximum airflow: xxx m³" ３ As an example, the case where " / min" is entered, "12 units" is entered for environmental control equipment F "lighting", and "3 units" is entered for environmental control equipment F "air purifier".

[0090] The third prompt generation unit 15C acquires equipment information by receiving the input equipment information from the input unit 12 and the output unit 13.

[0091] Furthermore, the third prompt generation unit 15C may acquire at least one of the following: equipment information, detection information, building information, and social information, by accepting input from the user. Alternatively, the third prompt generation unit 15C may acquire at least one of the following: equipment information, detection information, building information, and social information, from a predetermined data source. The predetermined data source is, for example, various types of data stored in an external server device connected to the information processing system 10 via a network NW or the like. The various types of data may include location information of windows and environmental adjustment equipment F, layout drawings within the target space TS, etc. That is, the third prompt generation unit 15C may acquire at least one of the following: equipment information, detection information, building information, and social information, from a predetermined data source by Retrieval Augmented Generation (RAG) based on instruction information that instructs the acquisition of at least one of these pieces of information.

[0092] The third prompt generation unit 15C then generates a third prompt that represents the acquired environmental element information and an instruction to generate control parameters for the environmental adjustment equipment F, which will realize an environmental state in the target space TS that users present in the target space TS will perceive as comfortable, in accordance with the environmental element information. The third prompt generation unit 15C can generate the third prompt, which includes the acquired environmental element information and the generation instruction, using a known method.

[0093] Returning to Figure 1, we continue the explanation.

[0094] The prompt generation unit 15D inputs a prompt to the LLM 25A that represents an instruction to generate control parameters for the environmental adjustment equipment F installed in the target space TS.

[0095] The prompt generation unit 15D generates prompts corresponding to at least two of the following: a first prompt, a second prompt, and a third prompt.

[0096] The prompt generation unit 15D selects any two or more prompts from the first prompt, the second prompt, and the third prompt. The prompt generation unit 15D then integrates the two or more selected prompts and generates a prompt that represents an instruction to generate control parameters for the environmental adjustment equipment F in order to realize the required environment represented by these prompts.

[0097] The prompts generated by the prompt generation unit 15D are prompts that represent adjustment request content for realizing a requested environment represented by at least two prompts, including a first prompt, a second prompt, and a third prompt.

[0098] Figure 3 is an explanatory diagram illustrating an example of processing by the prompt generation unit 15D.

[0099] For example, the prompt generation unit 15D acquires a first prompt generated by the first prompt generation unit 15A, a second prompt generated by the second prompt generation unit 15B, and a third prompt generated by the third prompt generation unit 15C. The prompt generation unit 15D then generates a prompt that includes these first, second, and third prompts, and an instruction to generate control parameters for the environmental adjustment equipment F to realize the requested environment represented by these prompts.

[0100] Specifically, for example, the prompt generation unit 15D generates prompts that include a first adjustment request content included in the first prompt, a second adjustment request content included in the second prompt, environmental element information included in the third prompt, and instructions for generating control parameters for the environmental adjustment equipment F to realize an environmental state in the target space TS that users present in the target space TS perceive as comfortable, in accordance with the first adjustment request content, the second adjustment request content, and the environmental element information.

[0101] The prompts generated by the prompt generation unit 15D only need to include at least two of the first, second, and third prompts, and are not limited to including all of the first, second, and third prompts.

[0102] The prompt generation unit 15D then inputs the generated prompt to the LLM 25A.

[0103] The LLM25A processes the input prompts to generate control parameters for the environmental adjustment equipment F, which is necessary to realize the requested environment represented by at least two prompts: a first prompt, a second prompt, and a third prompt.

[0104] Furthermore, LLM25A may generate reason information in addition to control parameters, indicating the reason for generating those control parameters. That is, LLM25A may generate control information that includes control parameters and reason information.

[0105] For example, consider a scenario where the prompt generation unit 15D receives a second prompt that includes the analysis results for the analysis request content "number of people, gender identification, age group, facial expression" for the captured image of the target space TS, and an instruction to generate control parameters according to the analysis results. Also consider a scenario where the prompt generation unit 15D receives a third prompt that includes environmental element information, including equipment information "number of air conditioning outlets and location of the air conditioning" and building information "type of building and number and location of windows included in the building," and an instruction to generate control parameters according to the environmental element information. Then, consider a scenario where the prompt generation unit 15D inputs the prompts, including the second and third prompts, to the LLM 25A and obtains the control parameters for the environmental adjustment equipment F from the LLM 25A. Furthermore, consider a scenario where the season in which the control of the environmental adjustment equipment F is performed is summer.

[0106] In this case, for example, the LLM25A generates control parameters that represent a lower air conditioning temperature the more users are included in the target space TS represented by the analysis results. Also, for example, the LLM25A generates control parameters that represent a higher air conditioning temperature the fewer users are included in the target space TS represented by the analysis results. Also, for example, the LLM25A generates control parameters that represent a higher air conditioning temperature the more the ratio of women to men in the target space TS represented by the analysis results is higher. Also, the LLM25A generates control parameters that represent a higher air conditioning temperature the older the age group of users included in the target space TS represented by the analysis results. In addition, the LLM25A generates control parameters that prevent the currently set air conditioning temperature from rising or falling too rapidly above a predetermined temperature.

[0107] Furthermore, LLM25A generates control parameters that, for example, represent a higher air conditioning temperature to allow users in the target space TS to sleep better, depending on their level of drowsiness as indicated by the analysis results, if the current time is the lunch break time for users in the target space TS. Also, LLM25A generates control parameters that, for example, represent a lower air conditioning temperature to allow users in the target space TS to wake up better, depending on their level of drowsiness as indicated by the analysis results, if the current time is the working time for users in the target space TS. Also, LLM25A generates control parameters that, for example, represent a lower level of drowsiness as indicated by the analysis results, if the number of users in the target space TS is below a threshold, to set the system to blow air from only some of the multiple air outlets, from an energy-saving perspective.

[0108] Furthermore, for example, if the building type represented by the building information is concrete, LLM25A generates air conditioning control parameters to repeat a cycle of 30 minutes of operation followed by a 10-minute shutdown. Also, for example, if the building type represented by the building information is wood, LLM25A generates air conditioning control parameters to repeat a cycle of 60 minutes of operation followed by a 10-minute shutdown. Furthermore, for example, as the outside air temperature decreases, LLM25A generates control parameters to operate in an energy-saving manner while introducing outside air into the room, according to the number and location of windows represented by the building information and the number and location of air conditioning outlets represented by the equipment information.

[0109] Furthermore, consider a scenario where the prompt generation unit 15D receives a second prompt, which includes the analysis result of the analysis request content "facial expression" for the captured image of the target space TS, and an instruction to generate control parameters according to the analysis result. Also consider a scenario where the prompt generation unit 15D receives a third prompt, which includes environmental element information, including building information "number and location of windows included in the building," and an instruction to generate control parameters according to the environmental element information. Then, consider a scenario where the prompt generation unit 15D inputs the prompts, including the second and third prompts, to the LLM 25A and obtains the control parameters of the environmental adjustment equipment F from the LLM 25A. Also consider a scenario where the season in which the control of the environmental adjustment equipment F is performed is summer.

[0110] In this case, for example, if the current time is during the working hours of users in the target space TS, LLM25A generates control parameters that represent brighter illumination as the drowsiness level of users in the target space TS, as expressed by the analysis results, increases. Also, for example, if the current time is during the lunch break of users in the target space TS, LLM25A generates control parameters that represent dimmer illumination to encourage sleep as the drowsiness level of users in the target space TS, as expressed by the analysis results, increases. Furthermore, for example, if the building has many windows, and the windows are positioned to easily let in natural light, LLM25A generates control parameters that represent dimmer illumination when the brightness outside the target space TS exceeds a threshold, from the perspective of saving energy by utilizing natural light. Also, for example, if the building has many windows, and the windows are positioned to easily let in natural light, LLM25A generates control parameters that represent turning off the lights located closer to the windows among multiple lights, from the perspective of saving energy, when the brightness outside the target space TS exceeds a threshold.

[0111] Returning to Figure 1, we continue the explanation.

[0112] The output control unit 15F outputs control information, including control parameters generated by the LLM 25A, to the output device 17. For example, the output control unit 15F displays the control information on the display screen 30.

[0113] Let's explain using Figure 2. For example, the output control unit 15F outputs the display screen 30 shown in Figure 2 to the output device 17. The display screen 30 includes a control information display area 34, as described above. The control information display area 34 is the display area for control information.

[0114] Figure 2 shows an example of a scene where the control parameters for an air conditioning system, an example of environmental control equipment F, namely the air conditioning control temperature "26°C", the lighting illuminance "5000 lx", and the lighting color temperature "3500 k", an example of lighting control parameters for an environmental control equipment F, are generated by the LLM25A and displayed in the control information display area 34. Figure 2 also shows an example of a scene where the control information display area 34 displays the reason information: "To provide a relaxing resting environment, the air conditioning is set to 26°C, the lighting to a soft light of 5000 lx, and the color temperature to 3500 k. Because it is a summer daytime and there are many people, this setting balances energy saving and comfort."

[0115] Returning to Figure 1, we continue the explanation.

[0116] The equipment control unit 15E controls the environmental adjustment equipment F based on the control parameters generated by the LLM 25A through the above processing by the prompt generation unit 15D. More specifically, the equipment control unit 15E controls the environmental adjustment equipment F so that it is set to the control parameters generated by the LLM 25A.

[0117] For example, consider a scenario where LLM25A generates control parameters such as an air conditioning control temperature of "26°C", lighting illuminance of "5000 lx", and lighting color temperature of "3500 k". In this case, the equipment control unit 15E sets the set temperature of the environmental control equipment F, which is the air conditioning system installed in the target space TS, to 26°C, and sets the illuminance of the environmental control equipment F, which is the lighting system installed in the target space TS, to 5000 lx and the color temperature to 3500 k. Through these controls, the environmental control equipment F installed in the target space TS is set to the set values ​​represented by the control parameters generated by LLM25A.

[0118] Therefore, by driving each of the environmental adjustment devices F installed in the target space TS to the set value, the environmental state of the target space TS is adjusted to the desired environment represented by the prompt.

[0119] Next, we will explain an example of the information processing flow performed in System 1.

[0120] Figure 4 is a sequence diagram showing an example of the information processing flow performed in System 1.

[0121] The first prompt generation unit 15A of the information processing system 10 generates a first prompt (step S100). The second prompt generation unit 15B of the information processing system 10 generates a second prompt (step S102). The third prompt generation unit 15C of the information processing system 10 generates a third prompt (step S104).

[0122] The prompt generation unit 15D of the information processing system 10 generates a prompt based on the two or more prompts generated in steps S100 to S104 (step S106). Then, the prompt generation unit 15D of the information processing system 10 inputs the generated prompt to the LLM 25A (step S108).

[0123] The LLM25A generates control information, including control parameters, based on the input prompt and outputs it to the information processing system 10 (steps S110, S112).

[0124] The equipment control unit 15E of the information processing system 10 controls the environmental adjustment equipment F based on the control parameters included in the control information (steps S114 and S116). Then, this sequence ends.

[0125] Figure 5 is a flowchart showing an example of the information processing flow performed by the information processing system 10 of this embodiment.

[0126] The first prompt generation unit 15A generates the first prompt (step S200). The second prompt generation unit 15B generates the second prompt (step S202). The third prompt generation unit 15C generates the third prompt (step S204).

[0127] The prompt generation unit 15D generates a prompt based on the two or more prompts generated in steps S200 to S204 (step S206). The prompt generation unit 15D then inputs the generated prompt to the LLM 25A (step S208). The LLM 25A generates control information including control parameters based on the input prompt and outputs it to the information processing system 10.

[0128] The equipment control unit 15E acquires control information generated by the LLM25A upon input of the prompt in step S208 to the LLM25A (step S210). The equipment control unit 15E controls the environmental adjustment equipment F based on the control parameters included in the control information acquired in step S210 (step S212). Then, this sequence ends.

[0129] As described above, the information processing system 10 of this embodiment includes at least one processor, which inputs a prompt representing an instruction to generate control parameters for the environmental adjustment equipment F installed in the target space TS to the LLM25A (language model), and controls the environmental adjustment equipment F based on the control parameters generated by the LLM25A.

[0130] In this embodiment, the information processing system 10 utilizes a language model (LLM25A) to input prompts representing instructions for generating control parameters for the environmental adjustment equipment F installed in the target space TS to the LLM25A (language model), and controls the environmental adjustment equipment F based on the control parameters generated by the LLM25A.

[0131] Therefore, in the information processing system 10 of this embodiment, even if the amount of reference data such as temperature data used to generate control parameters increases, equipment control for realizing the required environment can be easily performed with a simple configuration without requiring complex analysis and processing.

[0132] Therefore, the information processing system 10 of this embodiment can easily perform equipment control to realize the required environment with a simple configuration.

[0133] Furthermore, in the information processing system 10 of this embodiment, at least one processor outputs control parameters to the output device 17. Therefore, in the information processing system 10 of this embodiment, the control parameters used to realize the requested environment can be provided to the user in a verifiable manner.

[0134] Furthermore, in the information processing system 10 of this embodiment, at least one processor inputs prompts to the LLM25A (language model) corresponding to at least two of the following: a first prompt representing the content of a first adjustment request from the user to the environmental adjustment equipment F; a second prompt representing the content of a second adjustment request corresponding to the analysis result of at least one of the captured image and sound of the target space TS; and a third prompt including environmental element information (equipment information, building information, social information, current and past environmental measurement sensor data) related to the environmental elements of the target space TS.

[0135] In this embodiment, the information processing system 10 inputs prompts corresponding to multiple types of prompts to the LLM25A. Therefore, even when using prompts corresponding to multiple prompts, the information processing system 10 of this embodiment uses the LLM25A, and in addition to the above effects, it is possible to suppress the complexity of the system.

[0136] Furthermore, at least one processor of the information processing system 10 generates a first prompt corresponding to the content of the first adjustment request entered by the user.

[0137] Therefore, the information processing system 10 of this embodiment can perform equipment control to realize a desired environment that takes into account the user's requests regarding the environmental state of the target space TS.

[0138] Furthermore, at least one processor of the information processing system 10 generates a second prompt representing a second adjustment request based on the analysis results obtained by analyzing at least one of the captured images and audio of the target space TS, according to the analysis request. The analysis results also represent at least one of the attributes of the user (person) present in the target space TS and the state of the person.

[0139] Therefore, the information processing system 10 of this embodiment can perform equipment control according to the analysis results in accordance with the user's desired analysis request for at least one of the captured images and audio of the target space TS.

[0140] Furthermore, the environmental element information included in the third prompt includes at least one of the following: detection information detected by the environmental sensor S of the target space TS, equipment information relating to the environmental adjustment equipment F, building information relating to the buildings included in the target space TS, and social information relating to the target space TS.

[0141] Therefore, in the information processing system 10 of this embodiment, it is possible to easily perform equipment control of the environmental adjustment equipment F according to multiple types of information with a simple configuration.

[0142] Furthermore, the detection information includes at least one or more types of current and past detection results detected by the environmental sensor S.

[0143] By using detection information including current detection results, the information processing system 10 of this embodiment can perform equipment control of the environmental adjustment equipment F by feeding back the latest detection results. Furthermore, by using detection information including past detection results, the information processing system 10 of this embodiment can easily perform equipment control of the environmental adjustment equipment F according to multiple types of information with a simple configuration.

[0144] Furthermore, social information includes at least one of the following: preference trend information regarding users' preferences for the environment in the real space including the target space TS; weather information regarding the weather in the real space including the target space TS; age group information of users utilizing the target space TS; and employment status information of users utilizing the target space TS.

[0145] Therefore, in the information processing system 10 of this embodiment, it is possible to easily perform equipment control of the environmental adjustment equipment F according to multiple types of information with a simple configuration.

[0146] Furthermore, at least one processor of the information processing system 10 generates a third prompt that includes at least one of facility information, building information, and social information obtained from a predetermined data source.

[0147] Therefore, the information processing system 10 of this embodiment can utilize RAG and can further perform equipment control of the environmental adjustment equipment F according to multiple types of information.

[0148] The equipment information includes information on the specifications and at least one of the installation locations of the environmental control equipment F installed in the target space TS. The building information also includes information on at least one of the following: the number of windows, the arrangement of windows, the material, and the year of construction, which are included in the building that constitutes the target space TS or is installed in the target space TS.

[0149] Therefore, in the information processing system 10 of this embodiment, it is possible to easily perform equipment control of the environmental adjustment equipment F according to multiple types of information with a simple configuration.

[0150] Next, an example of the hardware configuration of the information processing system 10 and external system 20 of the above embodiment will be described.

[0151] Figure 6 is a hardware configuration diagram of an example of the information processing system 10 and external system 20 of the above embodiment.

[0152] The information processing system 10 and external system 20 in the above embodiment have a CPU (Central Processing Unit) 80, ROM (Read Only Memory) 82, RAM (Random Access Memory) 84, and I / F 86 etc. interconnected by a bus 88, and have a hardware configuration that uses a normal computer.

[0153] The CPU 80 is a arithmetic unit that controls the information processing system 10 and the external system 20 of the above embodiment. The ROM 82 stores programs and the like that realize information processing by the CPU 80. The RAM 84 stores data necessary for various processes performed by the CPU 80. The I / F 86 is an interface connected to the storage unit, input unit, output unit, sensor, and communication unit, etc., for sending and receiving data.

[0154] In the information processing system 10 and external system 20 of the above embodiment, the CPU 80 reads a program from the ROM 82 onto the RAM 84 and executes it, thereby realizing each of the above-mentioned functional units on the computer.

[0155] Furthermore, the programs for executing each of the above processes performed by the information processing system 10 and the external system 20 in the above embodiment may be stored in an HDD (hard disk drive). Alternatively, the programs for executing each of the above processes performed by the information processing system 10 and the external system 20 in the above embodiment may be pre-installed and provided in ROM 82.

[0156] Furthermore, the program for executing the above-described processing performed by the information processing system 10 and external system 20 of the above embodiment may be provided as a computer program product by being stored in an installable or executable file format on a computer-readable storage medium such as a CD-ROM, CD-R, memory card, DVD (Digital Versatile Disk), or flexible disk (FD). Alternatively, the program for executing the above-described information processing performed by the information processing system 10 and external system 20 of the above embodiment may be stored on a computer connected to a network such as the Internet and provided by being downloaded via the network. Alternatively, the program for executing the above-described information processing performed by the information processing system 10 and external system 20 of the above embodiment may be provided or distributed via a network such as the Internet.

[0157] Although embodiments have been described above, these embodiments are presented as examples only and are not intended to limit the scope of the invention. This novel embodiment can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. This embodiment and its variations are included in the scope and spirit of the invention, as well as in the claims of the invention and its equivalents.

[0158] Furthermore, this technology can also take the following configurations: (1) An information processing system comprising at least one processor, wherein the at least one processor inputs a prompt representing an instruction to generate control parameters for environmental adjustment equipment installed in a target space to a language model, and controls the environmental adjustment equipment based on the control parameters generated by the language model. (2) The information processing system according to (1), wherein the at least one processor outputs the control parameters to an output device. (3) The information processing system according to (1) or (2), wherein the at least one processor inputs a prompt to the language model corresponding to at least two of the following: a first prompt representing a first adjustment request content by a user for the environmental adjustment equipment; a second prompt representing a second adjustment request content corresponding to at least one analysis result of captured images, audio, and vital data from a wearable sensor of the target space; and a third prompt including environmental element information relating to environmental elements of the target space. (4) The information processing system according to (3), wherein the at least one processor generates the first prompt corresponding to the first adjustment request content input by the user. (5) The information processing system according to (3) or (4), wherein the at least one processor generates the second prompt representing the second adjustment request content according to the analysis result obtained by analyzing at least one of the captured image, the audio, and the vital data from the wearable sensor based on the content of the analysis request. (6) The information processing system according to any one of (3) to (5), wherein the analysis result represents at least one of the attributes of a person present in the target space and the state of a person. (7) The information processing system according to any one of (3) to (6), wherein the environmental element information includes at least one of detection information detected by the environmental sensor of the target space, equipment information relating to the environmental adjustment equipment, building information relating to a building included in the target space, and social information relating to the target space. (8) The information processing system according to (7), wherein the detection information includes at least one or more types of current and past detection results detected by the environmental sensor.(9) The information processing system according to (7) or (8), wherein the social information includes at least one of the following: preference trend information relating to the user's preference trends toward the environment in the real space including the target space; weather information relating to the weather in the real space including the target space; age group information of users utilizing the target space; and employment status information of users utilizing the target space. (10) The information processing system according to any one of (7) to (9), wherein the at least one processor generates the third prompt which includes at least one of the equipment information, the building information, and the social information obtained from a predetermined data source. (11) The information processing system according to any one of (7) to (10), wherein the equipment information includes information relating to at least one of the specifications and installation location of the environmental adjustment equipment provided in the target space. (12) The information processing system according to any one of (7) to (11), wherein the building information includes information relating to at least one of the number of windows, the arrangement of windows, the material, and the year of construction of a building that constitutes the target space or is provided in the target space. (13) The information processing system according to any one of (1) to (12), wherein the environmental adjustment equipment includes at least one of an air conditioning system, lighting fixtures, ventilation equipment, a humidifier, a fan, a water heater, a curtain opening / closing device, and an electric aroma pot. (14) The information processing system according to any one of (1) to (13), wherein the control parameters include information representing a set value for causing the environmental adjustment equipment to control at least one of the temperature, humidity, concentration of a specific chemical substance, odor, illuminance of lighting, color temperature of lighting, airflow, wind speed, and atmospheric pressure in the target space. (15) An information processing method performed by an information processing system, comprising inputting a prompt representing an instruction to generate control parameters for environmental adjustment equipment installed in a target space into a language model, and controlling the environmental adjustment equipment based on the control parameters generated by the language model. (16) An information processing program for causing a computer to input a prompt representing an instruction to generate control parameters for environmental adjustment equipment installed in a target space into a language model, and to control the environmental adjustment equipment based on the control parameters generated by the language model.

[0159] 10 Information Processing System 15 Control Unit 15A First Prompt Generation Unit 15B Second Prompt Generation Unit 15C Third Prompt Generation Unit 15D Prompt Generation Unit 15E Equipment Control Unit 15F Output Control Unit 17 Output Device 25A LLM

Claims

1. An information processing system comprising at least one processor, wherein the at least one processor inputs prompts representing instructions for generating control parameters for environmental adjustment equipment installed in a target space to a language model, and controls the environmental adjustment equipment based on the control parameters generated by the language model.

2. The information processing system according to claim 1, wherein at least one processor outputs the control parameters to an output device.

3. The information processing system according to claim 1, wherein the at least one processor inputs to the language model prompts corresponding to at least two of the following: a first prompt representing the content of a first adjustment request by a user to the environmental adjustment equipment; a second prompt representing the content of a second adjustment request corresponding to at least one analysis result of captured images, audio, and vital data from a wearable sensor of the target space; and a third prompt including environmental element information relating to environmental elements of the target space.

4. The information processing system according to claim 3, wherein the at least one processor generates the first prompt in response to the content of the first adjustment request entered by the user.

5. The information processing system according to claim 3, wherein the at least one processor generates the second prompt representing the second adjustment request content corresponding to the analysis result obtained by analyzing at least one of the captured image, the audio, and the vital data from the wearable sensor based on the analysis request content.

6. The information processing system according to claim 3, wherein the analysis results represent at least one of the attributes and state of a person present in the target space.

7. The information processing system according to claim 3, wherein the environmental element information includes at least one of detection information detected by environmental sensors in the target space, equipment information relating to the environmental adjustment equipment, building information relating to buildings included in the target space, and social information relating to the target space.

8. The information processing system according to claim 7, wherein the detection information includes at least one or more types of current and past detection results detected by the environmental sensor.

9. The information processing system according to claim 7, wherein the social information includes at least one of the following: preference trend information relating to the user's preference trends regarding the environment in the real space including the target space; weather information relating to the weather in the real space including the target space; age group information of users utilizing the target space; and employment status information of users utilizing the target space.

10. The information processing system according to claim 7, wherein the at least one processor generates the third prompt which includes at least one of the equipment information, the building information, and the social information obtained from a predetermined data source.

11. The information processing system according to claim 7, wherein the equipment information includes information relating to at least one of the specifications and installation location of the environmental control equipment provided in the target space.

12. The information processing system according to claim 7, wherein the building information includes information relating to at least one of the following: the number of windows, the arrangement of windows, the material, and the year of construction, which are included in the building that constitutes the target space or is provided in the target space.

13. The information processing system according to claim 1, wherein the environmental control equipment includes at least one of an air conditioning system, lighting fixtures, ventilation equipment, a humidifier, a fan, a water heater, a curtain opening / closing device, and an electric aroma pot.

14. The information processing system according to claim 1, wherein the control parameters include information representing a set value for causing the environmental control equipment to control at least one of the temperature, humidity, concentration of a specific chemical substance, odor, illuminance of lighting, color temperature of lighting, airflow rate, wind speed, and atmospheric pressure in the target space.

15. An information processing method performed by an information processing system, comprising: inputting a prompt representing an instruction to generate control parameters for environmental adjustment equipment installed in a target space into a language model; and controlling the environmental adjustment equipment based on the control parameters generated by the language model.

16. An information processing program for causing a computer to input prompts representing instructions for generating control parameters for environmental adjustment equipment installed in a target space into a language model, and to control the environmental adjustment equipment based on the control parameters generated by the language model.

Images