Information processing device, method for controlling the information processing device, and control program for the information processing device

Abstract

To provide information processing devices and the like that enable more flexible and optimized equipment control in building operations, while reducing the burden on administrators and achieving energy conservation and labor savings. [Solution] An information processing device according to one embodiment of the present invention includes an instruction receiving unit that receives setting instructions for a plurality of agents that interact using a language model, the plurality of agents being one or more first agents corresponding to one or more indicators indicating the status of a facility, one or more second agents relating to operational characteristics related to the operation of the facility, and one or more third agents that interact with the first and second agents regarding the operational policy of the facility and advance the interaction; a providing unit that provides information regarding the operational policy of the facility to the plurality of agents set according to the setting instructions; and an output processing unit that outputs the results of the interaction between the plurality of agents regarding the operational policy of the facility.

Description

Technical Field

[0001] The present invention relates to an information processing apparatus, a control method for the information processing apparatus, and a control program for the information processing apparatus.

Background Art

[0002] In recent years, so-called smart buildings (smart building) that analyze the flow of people and the environment inside and outside buildings by utilizing various information obtained from IoT (Internet of Things) sensors and air conditioning and lighting equipment, and achieve improved building operation efficiency and energy savings, have been spreading (for example, Patent Documents 1 and 2).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Summary of the Invention

Means for Solving the Problems

[0006] In an information processing device according to one embodiment of the present invention, the instruction receiving unit may receive instructions to set up a first agent, which evaluates a plurality of indicators based on at least one of the following: event information relating to the facility, human attributes and physical attributes, various information acquired by a plurality of types of information acquisition devices installed in the facility, the operating status of the facility's equipment, and external status information related to the facility obtained from a predetermined external information source, and engages in dialogue regarding the facility's operational policy.

[0007] In an information processing device according to one embodiment of the present invention, the instruction receiving unit may receive instructions to set up a second agent that engages in dialogue regarding the operational policy of the facility based on operational characteristics based on at least one of the control history of the facility's equipment and management records.

[0008] In an information processing device according to one embodiment of the present invention, the instruction receiving unit may receive an instruction to set up a third agent that facilitates a dialogue regarding the facility's operational policy using at least one of the costs related to the facility's operation, past operational records, and supply and demand forecasts.

[0009] In an information processing device according to one embodiment of the present invention, the output processing unit may summarize or partially extract and output the dialogue history between a plurality of agents according to predetermined conditions.

[0010] In an information processing device according to one embodiment of the present invention, the instruction receiving unit may further receive text input from a user for dialogue between multiple agents, and the third agent may proceed with the dialogue based on the content of the text.

[0011] In an information processing device according to one embodiment of the present invention, the instruction receiving unit may further receive input of predetermined conditions relating to the progress of dialogue between a plurality of agents, and the third agent may proceed with the dialogue based on the predetermined conditions.

[0012] In an information processing device according to one embodiment of the present invention, the instruction receiving unit may further receive setting instructions that cause multiple agents to output operational characteristics of the facility in accordance with the facility's operational policy as a dialogue result, and the output processing unit may output control signals to control the facility's equipment according to the operational characteristics of the facility included in the dialogue result.

[0013] A control method for an information processing device according to one embodiment of the present invention involves the information processing device receiving a setting instruction for a plurality of agents that interact using a language model, which include one or more first agents corresponding to one or more indicators indicating the state of a facility, one or more second agents relating to operational characteristics related to the operation of the facility, and one or more third agents that interact with the first and second agents regarding the operational policy of the facility and facilitate the interaction; providing information regarding the operational policy of the facility to the plurality of agents set according to the setting instruction; and outputting the results of the interaction between the plurality of agents regarding the operational policy of the facility.

[0014] A control program for an information processing device according to one embodiment of the present invention provides the information processing device with the following functions: a function to receive setting instructions for a plurality of agents that interact using a language model, including one or more first agents corresponding to one or more indicators indicating the status of a facility, one or more second agents relating to operational characteristics related to the operation of the facility, and one or more third agents that interact with the first and second agents regarding the operational policy of the facility and facilitate the interaction; a function to provide information regarding the operational policy of the facility to the plurality of agents set according to the setting instructions; and a function to output the results of the interaction between the plurality of agents regarding the operational policy of the facility. [Brief explanation of the drawing]

[0015] [Figure 1] Figure 1 shows an example of an information processing system configuration according to one embodiment of the present invention. [Figure 2] Figure 2 shows an example of a multi-agent configuration according to one embodiment of the present invention. [Figure 3] Figure 3 shows an example of the screen of an administrator terminal according to one embodiment of the present invention. [Figure 4] Figure 4 is an example of a flowchart of a control method for an information processing device according to one embodiment of the present invention. [Modes for carrying out the invention]

[0016] Hereafter, an embodiment of the invention disclosed herein (also referred to as the present invention) will be described in detail with reference to the drawings. Note that the drawings are examples only, and the present invention is not limited to those shown. For example, the illustrated information processing device, facilities, various equipment, information acquisition device, database, network, database server, administrator terminal, and number of agents, information processing system configuration, functional block diagram, screen examples, agent types, prompts (instructions), dialogue history, and flowcharts are examples only, and the present invention is not limited to these.

[0017] In this specification, generative AI is also known as generative AI, generative system AI, or generative AI, and refers to artificial intelligence (including AGI: Artificial General Intelligence or ASI: Artificial Superintelligence) that generates content such as text, images, audio, and video using deep learning technologies such as transformers, self-attention, and autoregressive networks, including language models (LLM: Large Language Model / SLM: Small Language Model), GPT (Generative Pre-trained Transformer, registered trademark), Gemini (registered trademark), Claude (registered trademark), Llama (registered trademark), and other language models. Extension technologies for generative AI include frameworks such as Retrieval-Augmented Generation (RAG), Memory-Augmented Generation, Hybrid Search using Vector Databases, Chunking / Chunk Processing, Knowledge Graph Linking, Entity Linking, AutoGen, AOG, and LangChain. Techniques for improving the performance of generative AI include fine-tuning using RLHF / RLAIF (Reinforcement Learning from Human Feedback / Reinforcement Learning with AI Feedback), PEFT (Parameter-Efficient Fine-Tuning), LoRA (Low-Rank Adaptation), distillation, quantization, weight sharing, continuous learning, associative learning, and in-context learning.Furthermore, the generating AI can operate in any environment, including on-premise, cloud, or edge (on-device), and parallel and distributed learning and inference using GPUs (Graphics Processing Units), TPUs (Tensor Processing Units), NPUs (Neural Processing Units), IPUs (Intelligence Processing Units), ASICs (Application-Specific Integrated Circuits), and FPGAs (Field-Programmable Gate Arrays).

[0018] Generative AI as used herein may include AI agents capable of acting autonomously with specific purposes or roles. Generative AI may constitute a so-called multi-agent AI through the cooperation, collaboration, or coordination of two or more AI agents. Generative AI may be applied to an extremely wide range of application areas, including call center support, automated FAQ responses via chatbots, AI assistants, translation, summarization, and meeting minute generation, programming assistance (code generation, debugging), data analysis, system development, financial and legal document review, research, diagnostics, drug discovery, design optimization, supply chain management, education, games, metaverse, advertising, creative and marketing, e-commerce (recommendations), threat intelligence, image / video / music generation, robotics, smart factories, smart cities, traffic control, autonomous driving, and IoT device control. In these application areas, generative AI can realize various services, businesses, or operations by integrating and executing multiple tasks. Use cases for generative AI include both internal processing types used in systems at the backend and chatbot types provided to users at the frontend.

[0019] As used herein, "orchestrator AI (also referred to as orchestration AI)" may mean an AI that assumes the role of coordinating, collaborating, and linking among multiple AI agents. The orchestrator AI may function as a command center as needed, and may perform allocation of processing to two or more AI agents, control of execution order, management of progress of dialogue, and integration of results. As a result, two or more AI agents can be chained and executed while collaborating, cooperating, and linking with each other, and can solve various tasks and problems as a common team. On the other hand, as issues with two or more AI agents collaborating, cooperating, and linking, risks such as hallucination, fake, bias, discrimination, data leakage, misdelivery, theft, privacy (personal information) protection, and copyright infringement may become apparent. Therefore, AI agent management and monitoring (logging, monitoring) technologies for ensuring security, governance, responsibility, auditing, reliability, and compliance may be used.

[0020] In the future, for the management and operation of AI agents, infrastructure such as high-performance GPUs, quantum computers, HPC (High Performance Computing) clusters, distributed / parallel processing, load balancers, CDNs (Content Delivery Networks), multi-access edge computing (MEC), and fog computing will be required. And AI agents are expected to acquire autonomy while achieving both reliability and economy (cost), and to develop to the AGI or ASI level beyond singularity. The present invention encompasses these future technological trends and application possibilities and is not limited to each embodiment described in this specification.

[0021] In recent years, due to labor shortages, the need to adapt to a decarbonized society, and the impact of the COVID-19 pandemic, there has been a strong demand for increased efficiency, reduced manpower, and optimized energy consumption in building operations. Traditionally, building operations have relied on central monitoring systems to control air conditioning and lighting equipment, as well as on monitoring-based control using IoT sensors such as temperature sensors, humidity sensors, pedestrian flow sensors, and image analysis cameras. However, these conventional technologies often depend on the building manager's experience and predetermined rule-based mechanisms, making it difficult to adequately adapt to changes in equipment and usage conditions. Furthermore, the burden on managers with specialized know-how is significant, and it has been difficult to respond flexibly to different usage conditions and environmental changes in each building. In light of this background, there is a need for a new system that enables more flexible and optimized equipment control in building operations, while reducing the burden on managers and achieving energy conservation and reduced manpower. According to one embodiment of the present invention, a building operation support system may be provided using generative AI (Artificial Intelligence). Specifically, multiple AI agents with different characteristics and roles that utilize language models (LLM: Large Language Model / SLM: Small Language Model) may be combined, and proposals regarding building operations may be generated through dialogue between these agents. This makes it possible to significantly shorten the learning period and provide specific and highly accurate equipment control proposals without relying on conventional rule-based control.

[0022] <System Configuration> FIG. 1 is a diagram showing a configuration example of an information processing system according to an embodiment of the present invention. The information processing system 600 may be a system that supports the operation of a facility using generative AI. Hereinafter, as an example of a facility to which the information processing system 600 is applied, an office building in which a plurality of companies or tenants reside will be described. However, the facility targeted by the present invention is not limited to this. The facility may be a building or the site of the building, and for example, it may be a commercial facility, a convention center, a school, a condominium, an accommodation facility (hotel, inn, etc.), an exhibition facility such as a museum or art museum, a performance facility such as a theater or concert hall, or their sites. Here, the operation of the facility may refer to activities that manage and control various facilities and environments within the facility to ensure safety, comfort, and efficiency. In one embodiment of the present invention, the facility may be a smart building. That is, for the operation of the facility, for example, various facilities such as air conditioning equipment 31, lighting equipment 32, moving equipment 33 such as elevators, security equipment 34 including entrance / exit gates and face recognition systems, power generation equipment 35 such as solar power generation and wind power generation are monitored and controlled, and information obtained from these facilities, IoT sensors, information acquisition devices 39 such as cameras is collected, and data such as temperature, humidity, illuminance, and human traffic is analyzed may be included.

[0023] The information processing system 600 may include a facility 30, a server (information processing device) 100, a manager terminal 300, and a database server 400. The facility 30 is a smart building as described above and may include at least any one of, for example, air conditioning equipment 31, lighting equipment 32, moving equipment 33, security equipment 34, power generation equipment 35, and information acquisition device 39. Information collected from these various facilities 31 to 35 and the information acquisition device 39 may be transmitted from the gateway 40 to the server 100 via the network 500. In FIG. 1, only one of each of the various facilities 31 to 35, the information acquisition device 39, and the gateway 40 is shown, but there may be a plurality of these. Further, the information processing system 600 may further include a central monitoring system (not shown) that controls the various facilities 31 to 35 of the facility 30.

[0024] Server 100 may be an information processing device that performs various processes related to the operation support services of facility 30 provided by the information processing system 600. Server 100 may include a communication unit 120, a control unit 110, and a storage unit 170. The communication unit 120 may be implemented as hardware such as a NIC (Network Interface Card), communication software, or a combination thereof. The communication unit 120 may have the function of sending and receiving information between Server 100 and various equipment 31-35 of facility 30 and an information acquisition device 39. In addition, the communication unit 120 may send and receive information between Server 100 and an administrator terminal 300.

[0025] The control unit 110 is typically a processor and may include a central processing unit (CPU), a microprocessor (MPU), a graphics processor (GPU), an application-based integrated circuit (ASIC), a programmable logic device (FPGA), etc. The control unit 110 may read a program stored in the memory unit 170 and execute instructions and code contained in the program to perform the functions and methods shown in each embodiment.

[0026] The control unit 110 may construct a multi-agent system. That is, the control unit 110 may include multiple agents (first agent 151, second agent 152, ..., nth agent 15n). In this embodiment, "agent" is a unit of software that operates autonomously according to a predetermined purpose and may have the function of making decisions based on information from the external environment or other agents and executing predetermined processes (tasks). In addition, in one embodiment of the present invention, the agent may be an AI agent based on a language model (LLM / SLM). An agent utilizing a language model may understand natural language and be capable of autonomous processing, judgment, and response. Hereafter, "agent" may be synonymous with "AI agent". Also, "language model" may simply be referred to as "LLM".

[0027] A multi-agent system may refer to a configuration in which multiple agents cooperate and communicate with each other to share processing responsibilities. In one embodiment of the present invention, agents may proceed with processing while interacting with each other. Here, interaction between agents may refer to a processing process in which each agent mutually refers to the information and analysis results it holds and collaboratively makes judgments, inferences, and decisions. This interaction may be carried out using structured data or unstructured data (including natural language text). When a multi-agent system is implemented on server 100, each agent (first agent 151, second agent 152, ... nth agent 15n) may be deployed on control unit 110 as a process, thread, virtual machine, or container. The number of agents may be set arbitrarily.

[0028] Although Figure 1 shows an embodiment in which agent 150 is integrated with server 100, the present invention is not limited to this. For example, agent 150 may be implemented in a system that provides the functions of a generated AI model via network 500. Examples of generated AI systems include "Gemini®" by Google Inc., and "GPT-4®" and "GPT-5®" by OpenAI, Inc. Alternatively, "Azure OpenAI Service," which makes AI models developed by OpenAI, Inc. available on Microsoft Azure® by Microsoft Inc., may be used as the generated AI system 300. Agent 150 may be implemented based on prompts provided to the generated AI system. A prompt is information for inputting instructions, questions, or conditions to the AI, and may include text, voice, images, or other input forms.

[0029] The control unit 110 may further include an instruction receiving unit 111, a provisioning unit 112, and an output processing unit 113. The instruction receiving unit 111 may receive setting instructions for a plurality of agents 150. That is, the instruction receiving unit 111 may receive prompts defining each agent 151, 152, ... for example, via the dashboard of the administrator terminal 300. As will be described later, the instruction receiving unit 111 may also receive text input from the administrator regarding the dialogue between the plurality of agents. The administrator may then participate in the discussion between the agents and present questions or additional conditions. The instruction receiving unit 111 may also receive input of predetermined conditions regarding the progress of the dialogue (e.g., number of statements, progress phase, termination conditions, etc.), and the operation proposal agent 153 may proceed with the dialogue based on these conditions. Each agent implemented in one embodiment of the present invention will be described later.

[0030] The provision unit 112 may provide information regarding the operational policy of the facility 30 to a plurality of agents 151, 152, ... configured according to the setting instructions. The operational policy of the facility is a direction for the operation of the facility 30 and may be a set of goals to be achieved. For example, the operational policy of the facility may include goals such as improving energy efficiency compared to last month, reducing electricity costs compared to the same period last year, and increasing the flow of people within the facility. These goals may not be limited to mere efficiency improvements or cost reductions, but may also reflect social demands such as improving comfort and convenience for facility users, alleviating congestion, ensuring safety, and even reducing greenhouse gas emissions and promoting the use of renewable energy from a sustainability perspective. These policies may change dynamically in response to environmental conditions (temperature, humidity, illuminance, noise, etc.), the content and scale of events held within the facility 30, weather and seasonal factors, electricity supply and demand conditions, market price fluctuations, the manager's judgment, management policies, and revisions to external regulations and laws, and are not limited to the examples listed.

[0031] As will be explained in detail later, each agent 151, 152, ... may collaborate through dialogue according to their respective assigned roles and verify information to derive operational characteristics that conform to the above-mentioned operational policy of facility 30. Here, "operational characteristics" may mean specific operating conditions and control settings applied to various equipment 31-35 in order to realize the said operational policy. In other words, operational characteristics may be properties that characterize the way in which various equipment 31-35 operate and are controlled. Operational characteristics may be, for example, the operating hours and settings of various equipment 31-35. The output processing unit 113 may output the results of the dialogue between multiple agents regarding the operational policy of facility 30. In other words, the output processing unit 113 may output operational characteristics that conform to the operational policy of facility 30 as a result of the dialogue. For example, if the operational policy is "reduce electricity costs compared to the same period last year," the operational characteristics may include specific numerical values ​​such as "turn off the air conditioning equipment from 5am to 6pm" and "do not use the west entrance / exit gates after 6pm."

[0032] Although Figure 1 shows only one server 100, it is not limited to this. In other words, each function of server 100, as described later, may be implemented by multiple servers. Furthermore, server 100 may be, for example, a distributed server system that operates in cooperation via a network, or a so-called cloud server. That is, server 100 is not limited to physical devices, but may also include servers configured virtually using software.

[0033] The administrator terminal 300 may be a communication terminal used by the administrator of facility 30. The administrator terminal 300 may have an application (hereinafter also simply referred to as "the app") installed for using the operation support service. Alternatively, installation of the application on the administrator terminal 300 is not mandatory, and the administrator may access the operation support service webpage from the administrator terminal 300 via a web browser or the like and send the necessary information to the server 100. The administrator terminal 300 may display a screen (dashboard) that shows various information related to the operation of facility 30. The administrator terminal 300 may send requests to the server 100 in response to input operations received from the administrator via the dashboard.

[0034] The database server 400 may store various types of information (data) used in the operation support service. For example, the database server 400 may store various types of information at facility 30, such as equipment-related information, environmental information, and management-related information. Here, in addition to this equipment-related information, environmental information, and management-related information, information including event information related to the facility, human attributes, physical attributes, various types of information acquired by multiple types of information acquisition devices installed at the facility, the operating status of the facility's equipment, and external status information obtained from predetermined external information sources may be collectively referred to as "facility-related information." Equipment-related information may be operational status and history information acquired from various types of equipment 31 to 35. Examples of equipment-related information include, but are not limited to, the set temperature, humidity, and airflow of the air conditioning equipment 31, the on / off status and illuminance level of the lighting equipment 32, the operating status, movement logs, anomaly detection logs, and usage count of the mobile equipment 33, the entry and exit history of the security equipment 34, and the amount of power generated and battery remaining charge of the power generation equipment 35. Furthermore, environmental information is information acquired from the information acquisition device 39 and includes, but is not limited to, temperature, humidity, illuminance, noise level, congestion level, and length of stay. Management-related information is information related to the operation and maintenance of the facility 30 and includes, but is not limited to, maintenance information such as electricity, gas, and water usage, equipment inspection schedules, failure history, and maintenance records, as well as operational schedule information such as reservations for meeting rooms and shared spaces and event schedules, and manuals. Management-related information also includes, but is not limited to, user authentication information, access control data, communication logs, audit logs, and data access history. Hereafter, the above-mentioned information used in the operation of the facility 30, and the information described later, may be collectively referred to as "facility-related information." This information may be collected at predetermined time intervals (for example, set between 5 and 60 minutes, but not limited to this) depending on the type of information.

[0035] The storage unit 170 may store various programs and data necessary for the operation of the server 100. The storage unit 170 may include, for example, a hard disk drive (HDD), a solid state drive (SSD), flash memory, etc. The storage unit 170 may also include memory that functions as a work area used for processing by the control unit 110.

[0036] <Embodiment> An embodiment of the present invention will be described using Figures 2 to 4. Figure 2 may be an example of a multi-agent system according to one embodiment of the present invention. Figure 3 may be an example of a dashboard displayed on an administrator terminal 300. Figure 4 may be an example of a flowchart relating to a control method for a server 100 according to one embodiment of the present invention. However, the present invention is not limited to these.

[0037] Referring to Figure 2, agent 150 may include an environmental analysis agent 151a, a human flow analysis agent 151b, a rule learning agent 152, and an operation proposal agent 153. The environmental analysis agent 151a and the human flow analysis agent 151b correspond to the first agent in the claims. The rule learning agent 152 and the operation proposal agent 153 correspond to the second agent and the third agent in the claims, respectively.

[0038] The first agent 151 (environmental analysis agent 151a, pedestrian flow analysis agent 151b) may correspond to one or more indicators that indicate the state of the facility. Indicators that indicate the state of the facility may refer to various quantitative or qualitative information for evaluating the operational status of the facility, such as the state of the environment and pedestrian flow within the facility 30. Indicators that indicate the state of the facility may include indicators related to the environment and indicators related to pedestrian flow, and the environmental analysis agent 151a and the pedestrian flow analysis agent 151b may correspond to each of these indicators. Furthermore, indicators that indicate the state of the facility may be evaluated based on at least one of the following: event information related to the facility, human attributes, physical attributes, various information acquired by multiple types of information acquisition devices installed in the facility, the operating status of the facility's equipment, and external situation information related to the facility obtained from predetermined external information sources.

[0039] Events held at a facility can be any kind of event or activity held at the facility. For example, events held at a facility may include seminars, workshops (English conversation, investment, career development courses, etc.), sales events, health promotion workshops (yoga, stretching, walking, etc.), concerts, lectures, comedy performances, rakugo (traditional Japanese storytelling), etc. Events may also include, for example, trade fairs, exhibitions, fairs, joint company information sessions, study abroad / college fairs, workshops, hackathons, business meetings, shareholder meetings, art exhibitions, film screenings, theatrical performances, sports competitions, e-sports competitions, disaster prevention drills, community exchange events, seasonal events (summer festivals, Christmas markets, etc.). These are just examples, and the types of events are not limited to these.

[0040] Human attributes may refer to the characteristics and attributes of people staying at the facility. Examples of human attributes may include age group, gender, whether they are domestic residents or foreign visitors, whether they are employees or visitors (whether they have access rights to the facility), length of stay, frequency of visits, occupation or affiliation, physical characteristics or conditions (presence or absence of disability, wheelchair use, etc.), behavioral patterns (travel routes and areas of stay, etc.), and purpose of use (purchasing, sightseeing, learning, business use, etc.). Physical attributes may refer to the characteristics of objects present at the facility. Examples of physical attributes may include the preservation conditions for exhibits at the aforementioned exhibitions and fairs (need for drying or maintaining humidity), type and use of objects (exhibits, goods, fixtures, materials, equipment, etc.), size and weight, material and structure, durability and preservation conditions (temperature and humidity control, vibration isolation, light isolation, dust isolation, etc.), installation location and portability (fixed / mobile, indoor / outdoor, etc.). Note that these are merely examples, and the types of human attributes are not limited to these.

[0041] The various types of information acquired by multiple information acquisition devices may include environmental sensor information such as temperature, humidity, CO2 concentration, PM2.5 concentration, and illuminance; ventilation equipment operation logs; control target values ​​and set values ​​via BEMS (Building Energy Management System); outdoor weather information (temperature, humidity, and wind speed); and pedestrian flow information from LiDAR, cameras, infrared sensors, and motion sensors. Furthermore, the various types of information may include noise levels, air quality information such as VOCs and carbon monoxide, water quality information, power consumption logs, gas and water usage, elevator and escalator operation logs, pedestrian flow data from Wi-Fi (registered trademark) and beacons, authentication logs from entrance / exit gates and card keys, external traffic information, and disaster information. Furthermore, comfort indicators calculated from this various information (such as PMV (Predicted Mean Vote) / PPD (Predicted Percentage of Dissatisfied) parameters, IAQ indicators, heat index (WBGT (Wet Bulb Globe Temperature)), infectious disease risk assessment indicators, etc.) may be used to evaluate indicators that show the condition of a facility.

[0042] The operating status of the facility's equipment may include information such as the current operating status, control target values, set values, capacity limits, and past operation logs of various equipment. It may also include entry and exit data obtained from entrance and exit gates. Furthermore, the operating status of the facility's equipment may include the layout and zoning diagrams of each floor within the facility, the operating status of power equipment (distribution boards, generators, storage batteries, etc.), the operating status of air conditioning and lighting equipment, the status of water supply and drainage equipment and hot water supply equipment, elevator and escalator operation information, operation logs of security and disaster prevention equipment (cameras, fire alarms, sprinklers, etc.), and even past maintenance and inspection history and failure records.

[0043] External information obtained from designated external sources may include weather information, disaster information (earthquake warnings, flood warnings, etc.), communication failure information, traffic information, event information for the area surrounding Facility 30, event information for nearby large facilities, and other information that may affect the operation of the facility.

[0044] The environmental analysis agent 151a may use this information as appropriate to analyze environmental conditions from air conditioning settings, temperature and humidity, weather information, etc. The pedestrian flow analysis agent 151b may also use this information as appropriate to analyze pedestrian flow from visitor numbers, event information, etc., and predict future pedestrian flow. The environmental analysis agent 151a and the pedestrian flow analysis agent 151b may handle this information separately or share it when performing their assigned tasks.

[0045] Thus, according to one embodiment of the present invention, a specialized agent may be assigned to each indicator showing the state of the facility. Therefore, it becomes possible to operate the facility with greater flexibility, capable of responding to various environmental conditions and pedestrian flow conditions that may occur in the facility 30.

[0046] The second agent (rule-learning agent) 152 may be an agent related to the operational characteristics of the facility's operation. Operational characteristics may be properties that characterize the operation and control of various pieces of equipment 31-35. Operational characteristics may be evaluated based on the facility's equipment control history and management records. The facility's equipment control history may include past setting change logs (time of day and reason for change), administrator comments and instruction logs, and correlation data between comfort scores and operation results. Management records may include unstructured documents such as daily reports, weekly reports, and manuals, as well as time-series correspondence information between BMS control history and environmental changes. Furthermore, management records may include feedback from administrators regarding past proposals made by agent 150. The rule-learning agent 152 may analyze this facility control history, operational rules, feedback on proposals, etc., and provide information regarding operational policies. The rule-learning agent 152 may learn from the control history and feedback on proposals each time facility control or operational proposals are made.

[0047] The third agent (operation proposal agent) 153 may engage in dialogue with the first agent 151 and the second agent 152 regarding the operational policy for facility 30 and facilitate the dialogue. Operational policies and facility-related information may be provided to the operation proposal agent 153 from the provision unit 112. The operation proposal agent 153 may derive proposals for the optimal operation of the facility based on information obtained from other agents, facility operation costs, past operation records, power supply and demand forecasts, etc. Facility operation costs may include the unit price of electricity consumption, amount of electricity generated, contracted power, percentage of renewable energy use, environmental costs based on CO2 emissions, etc. Past operation records may include information on the correlation between past facility use and the amount of electricity consumed at that time, various equipment manuals, troubleshooting, user comfort evaluations, maintenance history, equipment failure records, etc. Power supply and demand forecasts may include information on electricity shortages, market price fluctuation forecasts, seasonal factors and weather forecasts, information on large-scale events held in the surrounding area, etc. The operational proposal agent 153 may use this information to interact with other agents and generate proposals regarding the operation of building facilities.

[0048] Thus, according to one embodiment of the present invention, an agent that has learned the rules of each facility is set up. Therefore, it becomes possible to make proposals that are more likely to be adopted, in accordance with the operational rules specific to each facility. Furthermore, since the agent, acting as a coordinator to facilitate the dialogue, possesses the information necessary for operational proposals, it becomes possible to make rational and flexible proposals that take into account complex conditions without relying on human judgment.

[0049] Each of the agents described above may be implemented by prompts that define their roles and characteristics. The prompts defining each agent may, but are not limited to, the following. <Investment Proposal Agent> Prompt: You are a consultant with extensive experience and knowledge in building management. Based on the information and opinions gathered by each agent, generate proposals for improving the operation of building facilities (air conditioning, ventilation, lighting, etc.). Dialogue Strategy: Demonstrate leadership, treat each agent's arguments fairly, resolve conflicts and ambiguities, and arrive at the most satisfactory conclusion. <Environmental Analysis Agent> Prompt: You are an environmental analysis specialist (data analyst). Collect data on air conditioning settings, temperature and humidity, weather information, etc. for each zone, and analyze the environmental conditions. Dialogue Strategy: Briefly explain the situation and leave the next action to other agents. <Human Flow Analysis Agent> Prompt: You are a specialist in pedestrian flow analysis (data analyst). Collect visitor numbers (current and past history), event information, etc., analyze congestion and pedestrian flow, and predict future pedestrian flow. Dialogue Strategy: Create a database of numerical data and trends, and focus on supporting the decision-making of other agents. <Rule-Learning Agent> Prompt: You are a consultant with extensive experience and knowledge in building management, and are familiar with the unique rules of various buildings. Please collect and analyze manual operation logs from building managers, building equipment control / operation rules, customer feedback, etc. Dialogue strategy: Adopt a person-centered explanation style, such as saying, "This type of operation has been done before."

[0050] The operational characteristics generated as a result of the dialogue by agent 150 may be sent to the output processing unit 113. The output processing unit 113 may output the results of the dialogue between multiple agents regarding the facility's operational policy to the administrator terminal 300.

[0051] Thus, according to one embodiment of the present invention, multiple AI agents with different characteristics and roles that utilize a language model may be combined, and proposals regarding building operations may be generated through dialogue between these agents. This makes it possible to propose specific and highly accurate equipment control without relying on conventional rule-based control, while significantly shortening the learning period.

[0052] Figure 3 may be an example screen of an administrator terminal 300 according to one embodiment of the present invention. Screen D10 may be a dashboard that comprehensively displays various information about the facility 30 and allows the administrator to perform various setting operations on the facility 30 and configure multi-agent settings. Note that the figure is just an example and the present invention is not limited thereto.

[0053] Screen D10 may include a dialogue history area 11 that displays the dialogue history of each agent 151a, 151b, 152, and 153. The dialogue history area 11 may display dialogue results, including suggestions for operational characteristics. Suggestions for operational characteristics may be specific equipment setting instructions, such as "The congestion level on the 10th floor is high, so please lower the air conditioning by 2 degrees," or "Please turn off the lights in Zone B on the 3rd floor." This allows the administrator to appropriately control various equipment in facility 30 by referring to the dialogue results. The administrator may also send instructions to the server 100 from the administrator terminal 300 to control various equipment. The server 100 may then send control signals for various equipment to facility 30.

[0054] The dialogue history area 11 may include a text input area 12 for the administrator regarding the dialogue between agents. The entered text is sent from the administrator terminal 300 to the server 100, and the operation proposal agent 153 may further advance the dialogue based on the content of the text. For example, in the example in Figure 3, the administrator may enter a question such as "In that case, to what extent can costs be reduced?" or an instruction such as "Please estimate what would happen if it were shortened to 10 minutes" into the input area, prompting the agent to engage in further dialogue.

[0055] Thus, according to one embodiment of the present invention, an administrator can participate in a dialogue with an agent and guide the agent's discussion based on the administrator's interests and intentions. As a result, it becomes easier to delve deeper into facility operation considerations and compare alternatives. Furthermore, administrators can give instructions in natural language, enabling the provision of highly usable operational support services.

[0056] Figure 3 illustrates a case where the entire dialogue history of each agent is displayed in the dialogue history area 11. However, the output processing unit 113 may summarize and output the dialogue history between multiple agents according to predetermined conditions. For example, the level of dialogue history summarization may be set in advance by the administrator as a predetermined condition, and the dialogue history may be summarized based on that summarization level. Here, the summarization level may be a setting value that indicates the granularity of the amount of information when displaying the dialogue history. There may be multiple levels of summarization, such as a predetermined percentage of the entire dialogue history, such as 10%. Alternatively, the output processing unit 113 may partially extract and output according to predetermined conditions. Partial extraction may refer to partially cutting out the dialogue history. The predetermined conditions may be, for example, conditions that target extraction for only the conclusion, only opposing opinions, only specific numerical parts, etc. This allows the administrator to grasp the content at an appropriate granularity according to their needs.

[0057] Furthermore, for example, the dashboard may allow the setting of predetermined conditions regarding the progress of dialogue between multiple agents. These predetermined conditions regarding the progress of dialogue may be rules that define the development of the dialogue between agents. For example, predetermined conditions regarding the progress of dialogue may include the time allotted for the dialogue, the number of times each agent can speak, and a flow for the dialogue to progress in stages. An example of a dialogue flow setting may be phases such as "organize - confirm - conflict - adjust - agree". Furthermore, predetermined conditions for the progress of the dialogue may include the order of speaking and division of roles (e.g., the analysis agent always speaks first, and only the proposal agent makes proposals), the progress being limited to a specific theme, constraints requiring risk assessment and presentation of evidence, termination conditions for ending the dialogue (e.g., the agreement level reaching a predetermined threshold (e.g., support rate for the proposed agreement is 80% or higher), or each agent reaching a predetermined number of speaking times), control conditions based on system resources (e.g., the dialogue proceeding within a range where the time required to generate a response does not exceed a predetermined upper limit (e.g., within 3 seconds), or controlling the consumed computing resources (GPU usage, number of API calls, etc.) so as not to exceed predetermined upper limits), or the specification of the output format (table format, priority enumeration, etc.). The instruction receiving unit 111 of the server 100 may obtain predetermined conditions for the progress of the dialogue entered as a prompt on the administrator terminal 300. The operation proposal agent 153 may then proceed with the dialogue based on the predetermined conditions. This allows for a divergent dialogue and improves the accuracy of the suggestions derived from the dialogue.

[0058] In the above description, the dialogue results, including suggestions from the agent, are output to the administrator terminal 300, and the administrator controls various equipment in facility 30 from the administrator terminal 300. However, control signals to control the equipment in facility 30 may be output from the server 100 to facility 30 without going through the administrator terminal 300. Alternatively, if a central monitoring system for controlling the various equipment in facility 30 exists separately from the server 100, control signals may be output directly from the server 100 to the central monitoring system. This improves the immediacy and automation of control, reduces operational delays, prevents human error, and improves the efficiency of facility operations.

[0059] <Server control flowchart> The control method for the server 100 described above will be explained using the flowchart in Figure 4. First, the instruction receiving unit 111 of the server 100 may receive setting instructions for a plurality of agents that interact using a language model, including one or more first agents corresponding to one or more indicators indicating the status of the facility, one or more second agents relating to operational characteristics related to the operation of the facility, and a third agent that interacts with the first and second agents regarding the operational policy of the facility and facilitates the interaction (step P11). These setting instructions may be so-called prompts. The providing unit 112 may provide information regarding the operational policy of the facility to the plurality of agents configured in accordance with the setting instructions (step P12). The output processing unit 113 may output the results of the interaction between the plurality of agents regarding the operational policy of the facility (step P13).

[0060] The present invention has been described based on various drawings and embodiments, but it should be noted that those skilled in the art will find it easy to make various modifications and alterations based on this disclosure. Therefore, it should be noted that these modifications and alterations are within the scope of the present invention. For example, the functions included in each component, step, etc., can be rearranged in a logically consistent manner, and multiple components or steps, etc., can be combined into one, divided, or omitted or changed as necessary. Furthermore, the configurations shown in the above embodiments may be combined as appropriate.

[0061] For example, the above description explained a multi-agent system consisting of four agents: an environmental analysis agent 151a, a human flow analysis agent 151b, a rule learning agent 152, and an operation proposal agent 153. However, the number of agents is not limited to this, and more agents may be configured. Alternatively, the number of agents may be reduced. Furthermore, the processes described as being performed by each agent may be performed by other agents, or they may be shared among multiple agents. In addition, the above description explained a centralized configuration in which the operation proposal agent 153 acts as a coordinator and manages the progress of dialogue between agents. However, a multi-agent system may also be a distributed configuration in which there is no coordinator and agents interact with each other.

[0062] Furthermore, proposals from agents may be made at predetermined times, such as 6:00 or 12:00. Alternatively, they may be made at any time the agent deems necessary.

[0063] The programs of each embodiment of this disclosure may be provided stored in a storage medium readable by the information processing device. The storage medium may be a "non-temporary tangible medium" capable of storing the program. The program may include, for example, a software program or an information processing device program. When each functional unit of the server 100 is implemented by software, the server 100 may function as multiple agents by having a processor execute a program loaded into memory.

[0064] The program disclosed herein may be implemented using any programming language, such as JavaScript®, Python, C, Go, Swift®, Koltin®, or Java®.

[0065] According to each aspect of this disclosure described above, by providing 5G and beyond network technologies, we can contribute to achieving Sustainable Development Goal (SDG) 9, "Build resilient infrastructure, promote inclusive and sustainable industrialization and foster innovation." [Explanation of Symbols]

[0066] 100 Servers (Information Processing Devices) 110 Control Unit 111 Instruction Reception Department 112 Provision Department 113 Output Processing Unit 120 Communications Department 150 agents 151 First Agent 152 Agent 2 15n The Nth Agent 151a Environmental Analysis Agent 151b Human Flow Analysis Agent 152 Rule-Learning Agents 153 Investment Proposal Agent 170 Storage section 300 Administrator terminals 400 Database Servers 500 Networks 600 Information Processing Systems 30 facilities 31 Air conditioning equipment 32 Lighting equipment 33 Mobile equipment 34 Security equipment 35 Power generation facilities 39 Information acquisition device 40 Gateways

Claims

1. As multiple agents that interact using language models, One or more first agents autonomously collect and analyze one or more indicators representing the status of the facility based on their respective roles, and provide the evaluation results in the aforementioned dialogue. One or more second agents autonomously collect and analyze at least one of the control history and management records of the equipment of the said facility, and provide information regarding the operational characteristics related to the operation of the said facility in dialogue with other agents, An instruction receiving unit that receives setting instructions for one or more third agents, which generate improvement proposals regarding the operational policy of the facility based on the information and opinions collected by the first agent and the second agent, respectively, and which facilitate dialogue with the first agent and the second agent, A provisioning unit that provides information regarding the operational policy of the facility to the plurality of agents configured in accordance with the setting instructions, An information processing device comprising an output processing unit that outputs the results of a dialogue between the multiple agents regarding the operational policy of the facility.

2. The instruction receiving unit receives setting instructions for the first agent, which correspond to at least one of the indicators related to the environment and the indicators related to human flow. The information processing apparatus according to claim 1.

3. The instruction receiving unit evaluates one or more indicators indicating the status of the facility based on at least one of the following: various information acquired by multiple types of information acquisition devices installed in the facility, event information related to the facility, human attributes and physical attributes, the operating status of the facility's equipment, and external status information related to the facility obtained from a predetermined external information source, and receives setting instructions for the first agent to engage in dialogue regarding the operational policy of the facility. The information processing apparatus according to claim 1.

4. The instruction receiving unit receives instructions to set up the third agent, which will proceed with a dialogue regarding the operational policy of the facility using at least one of the costs related to the operation of the facility, past operational records, and supply and demand forecasts. The information processing apparatus according to claim 1.

5. The output processing unit summarizes or partially extracts the dialogue history between the multiple agents according to predetermined conditions and outputs it. The information processing apparatus according to claim 1.

6. The instruction receiving unit further receives text input from the user for the dialogue between the multiple agents, The third agent proceeds with the dialogue based on the content of the text. The information processing apparatus according to claim 1.

7. The instruction receiving unit further receives input of predetermined conditions regarding the progress of dialogue between the multiple agents, The third agent proceeds with the dialogue based on the predetermined conditions. The information processing apparatus according to claim 1.

8. The instruction receiving unit further receives setting instructions that cause the multiple agents to output the operational characteristics of the facility in accordance with the operational policy of the facility as the dialogue result. The output processing unit outputs a control signal to control the equipment of the facility according to the operational characteristics of the facility included in the dialogue result. The information processing apparatus according to claim 1.

9. Information processing device, As multiple agents that interact using language models, One or more first agents autonomously collect and analyze one or more indicators representing the status of the facility based on their respective roles, and provide the evaluation results in the aforementioned dialogue. One or more second agents autonomously collect and analyze at least one of the control history and management records of the equipment of the said facility, and provide information regarding the operational characteristics related to the operation of the said facility in dialogue with other agents, A step of receiving instructions to configure one or more third agents, which will facilitate dialogue with the first and second agents while generating improvement proposals regarding the operational policies of the facility based on the information and opinions collected by the first and second agents respectively, The steps include providing information regarding the facility's operational policy to the plurality of agents configured according to the aforementioned setting instructions, The steps include outputting the results of the dialogue between the multiple agents regarding the operational policy of the facility, A method for controlling an information processing device to perform the following actions.

10. In an information processing device, As multiple agents that interact using language models, One or more first agents autonomously collect and analyze one or more indicators representing the status of the facility based on their respective roles, and provide the evaluation results in the aforementioned dialogue. One or more second agents autonomously collect and analyze at least one of the control history and management records of the equipment of the said facility, and provide information regarding the operational characteristics related to the operation of the said facility in dialogue with other agents, A function to receive setting instructions for one or more third agents that facilitate dialogue with the first and second agents while generating improvement proposals regarding the operational policies of the facility based on the information and opinions collected by the first and second agents, respectively, A function to provide information regarding the operational policy of the facility to the plurality of agents configured in accordance with the aforementioned setting instructions, A function to output the results of the dialogue between the multiple agents regarding the operational policy of the said facility, A control program for an information processing device that makes this possible.

Images