Building energy control method by llm-ai based on natural language processing

Abstract

The present invention relates to a method for real-time energy control through the interaction of an HMI, an AI, and a user utilizing Building Information Model (BIM)-based information, comprising the steps of: an information collection module receiving spatial information and equipment configuration information from the Building Information Model (BIM) and transmitting it to an AI module; an HMI module periodically transmitting building environmental status information and control history collected based on real-time sensor data to an AI module in JSON format, and the AI ​​module logging and analyzing the building environmental status information and control history as training data; an HMI module receiving a user query regarding a control request in natural language form and transmitting it to an AI module; an AI module interpreting the user query based on a Large Language Model (LLM), integrating and analyzing the interpreted query with the spatial information and environmental status information to infer a control objective, generating a control scenario, and providing a control recommendation to the user; and a user verifying the control recommendation provided by the AI ​​module and approving or instructing the HMI module to supplement the control recommendation. The AI ​​module receives user approval or supplementary instructions regarding a control recommendation and transmits the finalized control plan approved by the user to the HMI module, or receives the user's supplementary instructions to supplement the control recommendation and provides the supplemented finalized control plan to the HMI module; the HMI module receives the finalized control plan, generates a control signal, and transmits a control command to the relevant equipment to enable control execution; and the HMI module transmits the control execution results and user feedback to a feedback learning module.

Description

Technology Field

[0001] The present invention relates to a building energy control method, and more specifically, to a building energy optimization control method based on natural language processing-based LLM-AI that enables intuitive and efficient energy management through a natural language processing-based interface and optimizes energy control in real time through the interaction of HMI, AI, and user by utilizing Building Information Model (BIM)-based information. Background Technology

[0003] Generally, a Building Automation System (BAS) is a system that creates a comfortable environment by automatically controlling heating, cooling, power, lighting, air conditioning, and service facilities within a building, and promotes the efficient utilization of energy through integrated management. A Building Energy Management System (BEMS) is a system that monitors real-time energy usage to provide optimized energy management solutions for the purpose of maintaining a comfortable indoor environment and managing energy efficiently, and it is expected to enable practical management of personnel and reduce overall operating costs.

[0004] Since energy savings can be expected from using such BAS or BEMS, the number of buildings equipped with BAS and BEMS is gradually increasing.

[0005] As such, the system includes basic components of various sensors installed in each facility part of the building to detect temperature, humidity, illuminance, etc., a controller that generates operation signals after receiving signals detected by the sensors, an operation unit that executes control operations upon receiving operation signals from the controller, an information collection unit that collects and manages all of their information, a data transmission unit that transmits the collected information to the central processing unit, and a central processing unit that comprehensively analyzes and controls the collected information.

[0006] Conventional systems focus on energy minimization and efficiency optimization based on equipment-centric sensor data, lacking a configuration for interpreting user natural language queries and integrating building information, including spatial and equipment configurations, as well as environmental status data. Furthermore, control methods rely on fixed logic such as human intervention, threshold-based conditional branching, and scheduling, making it difficult to respond flexibly to real-time environmental changes and resulting in a lack of linkage between prediction, cost, and efficiency.

[0007] Furthermore, conventional systems suffer from a lack of automatic control scenario generation and supplementary frameworks based on user feedback, are limited to standardized or pattern learning methods, and struggle to reflect subjective user requests, resulting in reduced user comfort. Additionally, they have issues such as a lack of flexibility due to being based on fixed logic or limitations of manual control methods.

[0008] Furthermore, conventional systems struggle to respond in real-time to changes in the operating environment, such as weather, personnel, and schedules. Existing Human Machine Interfaces (HMIs) are limited to simple data display, making it difficult to interpret control logic. Additionally, there is a problem in that real-time optimized energy control cannot be performed through a cyclic learning structure involving natural language query interpretation, integrated analysis, and the learning and application of user feedback.

[0009] Meanwhile, regarding prior art related to this, Patent Document 1 discloses a building energy management system that collects power consumption data of power-using facilities, checks real-time power consumption, predicts subsequent power consumption, and performs peak power control based on the prediction, thereby maintaining the stability of the power grid. Prior art literature

[0011] Republic of Korea Published Patent Application No. 10-2025-0083935 (Published June 10, 2025) The problem to be solved

[0012] The present invention has been devised to solve the aforementioned problems and aims to provide a control method having an interconnected cyclic structure of BIM-HMI-AI-User (Operator), capable of interpreting natural language-based queries input by a user, integrating and analyzing them together with spatial and facility configuration information and real-time environmental status information to automatically generate and execute optimal control scenarios.

[0013] Furthermore, the present invention aims to provide a building energy optimization control method based on natural language processing LLM-AI, which enables accurate and consistent automatic control based on AI control recommendations compared to conventional fixed logic and manual control methods; allows for optimal control and anomaly detection by reflecting real-time conditions; features a natural language query interpretation and feedback loop structure; learns based on AI result interpretation and user feedback; enables integrated analysis of user natural language queries and judgment based on user intent, unlike conventional methods where reflecting user intent was difficult; features a self-optimizing system based on integrated analysis and learning, moving away from conventional standardized equipment-centric control; facilitates easy expansion by linking with BEMS / FEMS, etc., through a modular interlocking structure; saves energy through a feedback loop learning structure; and reduces user judgment fatigue and minimizes work errors. means of solving the problem

[0015] The present invention relates to a method for real-time energy control through the interaction of a Human Machine Interface (HMI), AI, and a user utilizing Building Information Model (BIM)-based information, comprising the steps of: an information collection module receiving spatial information and equipment configuration information from the Building Information Model (BIM) and transmitting it to an AI module; an HMI module periodically transmitting building environmental status information and control history collected based on real-time sensor data to an AI module in JSON (JavaScript Object Notation) format, and the AI ​​module logging and analyzing the building environmental status information and control history as training data; an HMI module receiving a user query regarding a control request in natural language form and transmitting it to an AI module; an AI module interpreting the user query based on a Large Language Model (LLM), integrating and analyzing the interpreted query with the spatial information and environmental status information to infer a control objective, generating a control scenario, and providing a control recommendation to the user; and a user verifying the control recommendation provided by the AI ​​module and approving or instructing the HMI module to supplement the control recommendation. A building energy control method based on natural language processing LLM-AI is provided, comprising the steps of: an AI module receiving a user's approval or supplementary instruction regarding a control recommendation, transmitting the finalized control plan approved by the user to an HMI module, or receiving the user's supplementary instruction to supplement the control recommendation and providing the supplemented finalized control plan to the HMI module; an HMI module receiving the finalized control plan, generating a control signal, and transmitting a control command to the corresponding facility to enable control execution; and an HMI module transmitting the control execution result and user feedback to a feedback learning module.

[0016] delete Effects of the invention

[0017] According to the present invention, based on a Large-Scale Language Model (LLM), a user's natural language query is interpreted and integrated with building environmental status information periodically collected based on real-time sensor data and building information including space and facility configurations received from a Building Information Model (BIM) to infer control objectives, generate control scenarios, and provide control recommendations, thereby enabling energy optimization control for facilities. Moving away from conventional fixed, static logic-based control methods, the creation of AI-based control scenarios tailored to the user's objectives and the provision of control recommendations improve the accuracy and consistency of energy control, reduce energy consumption, decrease user judgment fatigue, prevent errors, enhance user comfort and satisfaction, and improve responsiveness to anomaly detection. Furthermore, real-time energy optimization control and efficient operation are possible through a cyclic learning structure involving natural language query interpretation, integrated analysis, and user feedback learning and application. Additionally, the AI ​​module integrates and analyzes the user's natural language query and real-time environmental status information to infer control objectives, and judgment criteria are self-learned and advanced through repeated user feedback learning, gradually optimizing to meet the user's intent. Moreover, the system can be linked in a modular form with external systems such as BEMS / FEMS, is easy to introduce and expand, and enables the execution of integrated control. There is a possible effect. Brief explanation of the drawing

[0019] FIG. 1 is a flowchart of a control method according to an embodiment of the present invention. FIG. 2 is a flowchart illustrating an AI-based energy control HMI implementation algorithm according to an embodiment of the present invention. Figure 3 illustrates the code for system initialization and the main control loop according to an embodiment of the present invention. FIG. 4a shows spatial information and facility configuration information in the gbXML file format collected from a Building Information Model (BIM) according to an embodiment of the present invention; FIG. 4b shows JSON format data in which an HMI module according to an embodiment of the present invention transmits building environmental status information and control history to an AI module; FIG. 4c shows a control recommendation of an AI module according to an embodiment of the present invention in JSON format; FIG. 4d shows JSON format data regarding control execution results and user feedback according to an embodiment of the present invention; and FIG. 4e shows JSON format data for executing a feedback-based control confirmation plan according to an embodiment of the present invention. FIG. 5a is a configuration diagram of a system according to an embodiment of the present invention, and FIG. 5b is a configuration diagram of an AI module according to an embodiment of the present invention. FIG. 6 is a schematic diagram illustrating an AI-based energy control HMI mechanism according to an embodiment of the present invention. Specific details for implementing the invention

[0020] Hereinafter, with reference to the drawings, a building energy optimization control method based on natural language processing LLM-AI according to the present invention will be described in detail, focusing on embodiments.

[0021] Referring to FIGS. 1 to 3, the building energy control method by natural language processing-based LLM-AI according to the present invention enables intuitive and efficient energy management through a natural language processing-based interface and optimizes energy control in real time through the interaction of HMI, AI, and user by utilizing information based on a Building Information Model (BIM). First, an information collection module (100) receives spatial information and equipment configuration information from the Building Information Model (BIM) and transmits it to an AI module (300) (step S10).

[0022] The information collection module (100) receives spatial information and equipment configuration information in the gbXML file format from BIM and transmits it to the AI ​​module (300), and the AI ​​module (300) extracts spatial information and equipment configuration information from the data in the gbXML file format and loads it into the system (1). At this time, the spatial information includes a space ID, name, area, estimated number of personnel, equipment load, etc., and the equipment configuration information includes an equipment ID, type of equipment (HVAC, lighting, power, etc.), list of equipment by space, list of controllable equipment, location coordinates, connection zone, etc., and is information transmitted only once. For example, referring to Figure 4a, which shows spatial information and equipment configuration information in the gbXML file format collected from a Building Information Model (BIM), it provides spatial information for Office 1 with the ID "Office1", the number of personnel in the space is 7, and the HVAC system with the ID "FCU01" is of the "Fan Coil Unit" type and is connected to the "Office1" space (connection zone), and indicates that this HVAC system provides cooling and heating functions.

[0023] After step S10, the HMI module (200) periodically transmits the building's environmental status information and control history collected based on real-time sensor data to the AI ​​module (300), and the AI ​​module (300) performs a step (step S20) of logging and analyzing the building's environmental status information and control history as training data. The HMI module (200) periodically collects environmental status information, such as the building's temperature, humidity, illuminance, and occupancy count, based on real-time sensor data and transmits it to the AI ​​module (300) in JSON format.

[0024] Referring to FIG. 4b, which shows JSON format data in which an HMI module (200) according to an embodiment of the present invention transmits building environment status information and control history to an AI module (300), there is a control request from a user in the "Office 1" space saying "Cool down Office 1," and the current environment status information of Office 1 is a temperature of 26.8℃, humidity of 58%, and occupancy of 7 people. It also indicates that "FCU01" (Fan Coil Unit), which has cooling and heating functions, is located in "Office 1" as a controllable facility, and "LIGHT01" (lighting), which has on, off, and brightness adjustment functions, is located in "Office 1." At this time, the controllable facilities represent a list of facilities connected to the HMI module (200) for each space, and include location coordinates, equipment type, etc.

[0025] After step S20, the HMI module (200) receives a user's query regarding a control request in the form of natural language and transmits it to the AI ​​module (300) to request confirmation of whether the control currently being performed is an optimized control (step S30). The user's query can be input into the HMI module (200) in voice or text, and can be input into the HMI module (200) in voice or text by connecting wirelessly or wired to the system (1) through a manager module (400) such as a desktop PC, laptop, smartphone, tablet PC, smartwatch, PDA.

[0026] After step S30, the AI ​​module (300) performs a step (step S40) in which it interprets the user's query based on a large-scale language model (LLM), integrates and analyzes the interpreted query with the spatial information and environmental state information to infer the control objective, generates a control scenario, and provides a control recommendation to the user. The AI ​​module (300) interprets the user's query based on a large-scale language model (LLM), identifies the user's question intent based on prior learned knowledge, generates a scenario based on the user's subjective objective, and improves the precision of interpreting natural language queries. The AI ​​module (300) integrates and analyzes the interpreted query with information on spatial information and environmental state to automatically infer the control objective, generates a control scenario, and provides a control recommendation to the user.

[0027] At this time, when the AI ​​module (300) generates a control scenario, it can search for feedback data for the same conditions through the feedback learning module (500), and generate a feedback-based control scenario by applying a judgment criterion corrected by the searched feedback data, thereby providing a control recommendation to the user.

[0028] For example, referring to FIG. 4b, when a user's query is entered in natural language as "make Office 1 comfortable," the AI ​​module (300) interprets the query and integrates and analyzes spatial information and environmental status information of Office 1. When the current temperature of Office 1 is 26.8 degrees and the number of occupants is 7, the judgment criteria are that cooling is required when the number of occupants is 5 or more, and the target temperature is set to 22.5 degrees when the current temperature exceeds 26 degrees. Through this, the AI ​​module (300) can infer a control objective that cooling is required for Office 1, and by applying the judgment criteria, generates a control scenario to cool the target temperature of Office 1 to 22.5℃ and provides it to the user as a control recommendation along with the reason for the judgment.

[0029] FIG. 4c shows a control recommendation of an AI module (300) according to an embodiment of the present invention in JSON format. Referring to FIG. 4c, the space to be controlled is "Office 1", the control operation is "turn on cooling", the target temperature is set to "22.5℃", the reason for judgment is "current temperature is 26.8℃ and comfortable temperature is set according to user request", and the reliability of the information is 92%.

[0030] After step S40, the user checks the control recommendation provided by the AI ​​module (300) and performs a step (step S50) of approving or instructing the HMI module (200) to supplement the control recommendation. If the user approves the control recommendation, the control recommendation becomes the final confirmed control plan, and if the user instructs the AI ​​module (300) to supplement the control recommendation, the plan supplemented by the AI ​​module (300) becomes the final confirmed control plan. The results of such control execution can be fed back and reflected in the next control, and the user can approve, instruct supplement, or transmit user feedback to the system (1) through the administrator module (400).

[0031] After step S50, the AI ​​module (300) receives the user's approval or supplementary instructions regarding the control recommendation, transmits the control confirmation plan approved by the user to the HMI module (200), or receives the user's supplementary instructions to supplement the control recommendation and performs the step (step S60) of providing the supplemented control confirmation plan to the HMI module (200).

[0032] Referring to FIG. 4d, which shows JSON format data regarding the control execution result and user feedback, the feedback learning module (500) generates an Interaction ID "xyz789" for feedback identification by combining the user's query and environmental status information, and the control target space is "Office 1" and the user's query is entered as "Cool Office 1" in natural language, so the AI ​​module (300) starts cooling Office 1 through the "FCU01" equipment and sets the target temperature to 22.5℃, and as a result of the control execution, the final temperature of Office 1 is 23.5℃ and it operated for 40 minutes, and in response to this, the user provides feedback saying "Make it a little cooler," and the AI ​​module (300) indicates that the target temperature of the next recommendation is adjusted to 22.0℃ by reflecting the user's feedback, and the control execution result and user feedback are stored as the Interaction ID "xyz789".

[0033] After step S60, the HMI module (200) receives the confirmed control plan, generates a control signal, and transmits a control command to the corresponding equipment to enable the execution of optimized control (step S70). For example, referring to FIG. 4e, when the AI ​​module (300) generates a feedback-based scenario reflecting the feedback data of FIG. 4d and a confirmed control plan ("FCU01" (Fan Coil Unit) equipment, turn on cooling, target temperature 22.0℃) is prepared, the HMI module (200) generates a control signal for the confirmed control plan and transmits a control command to "FCU01" (Fan Coil Unit) to enable the execution of optimized control.

[0034] After step S70, the HMI module (200) can perform a step (step S80) of transmitting control execution results and user feedback to the feedback learning module (500), and a step (step S90) in which the feedback learning module (500) generates an Interaction ID for feedback identification by combining user queries and environment status information, and collects and stores control scenarios from the AI ​​module (300), control execution results and user feedback from the HMI module (200) based on the Interaction ID for learning. As a result, feedback is accumulated so that conditional judgment criteria are automatically corrected, execution results are fed back to the AI ​​module (300) to be reflected in future judgments, and through repeated user feedback learning, judgment criteria are refined to provide a cyclic structure that can be gradually optimized to match the user's intentions.

[0035] In addition, FIG. 2 is a flowchart illustrating an AI-based energy control HMI implementation algorithm according to an embodiment of the present invention described above, and FIG. 3 is a diagram explaining a main control loop according to an embodiment of the present invention described above.

[0036] Referring to FIGS. 2 and 3, as a system initialization, spatial information and facility configuration information in the gbXML file format are loaded into the system (1), and feedback data is loaded. Then, as a main control loop, a user query is received, building environmental status information and control history are collected based on real-time sensor data, the user query is interpreted based on a large-scale language model (LLM) to identify the user's intent, spatial information and facility configuration information such as a list of controllable facilities are verified, and an Interaction ID for feedback identification is generated by combining the user query and environmental status information.

[0037] In addition, if previously recorded feedback data does not exist, a scenario is generated based on the LLM's prior learned knowledge, and if feedback data exists, a feedback-based control scenario is generated by reflecting the retrieved feedback data.

[0038] The control recommendation includes the space name, control facility ID, control command, target value, reason for judgment, and judgment reliability, and executes the control command.

[0039] Next, control execution results and user feedback are collected, user queries, control scenarios, control execution results, and user feedback are stored based on the Interaction ID, and updated to enhance judgment criteria so that they are reflected in future decisions.

[0040] And, the terms described in Fig. 3 are briefly explained as follows.

[0041] user_query : Natural language query entered by the user

[0042] state: Real-time status information

[0043] intent: Identifying the intent behind LLM-based user queries

[0044] zone_info : Spatial (zone) metadata

[0045] available_devices : List of controllable facilities

[0046] interaction_id: A unique key generated based on the combination of the current query and state (for feedback identification)

[0047] feedback_data: User feedback and execution results saved under the same conditions in the past

[0048] scenario: AI-generated control scenario

[0049] recommendation: Control recommendations provided to the user

[0050] result : Control execution result

[0051] feedback : User responses or additional inquiries

[0052] In addition, the effects of the present invention are compared with those of the prior art as follows.

[0053]

[0054] In addition, the building energy control system (1) based on natural language processing-based LLM-AI according to the present invention enables intuitive and efficient energy management through a natural language processing-based interface and optimizes energy control in real time through the interaction of the AI ​​and the user with the Human Machine Interface (HMI) utilizing building information model (BIM)-based information. Referring to FIGS. 5a and 6, it comprises some or all of an information collection module (100), an HMI module (200), an AI module (300), a manager module (400), a feedback learning module (500), and an API module (600).

[0055] The information collection module (100) receives zone information and facility configuration information from Building Information Modeling (BIM), which is a building information model, and transmits it to the AI ​​module (300). BIM is a technology that integrates and manages information at all stages, such as design, construction, and maintenance, throughout the lifecycle of facilities in the construction field. The information collection module (100) receives zone information and facility configuration information in gbXML file format from BIM and transmits it to the AI ​​module (300), and the AI ​​module (300) extracts zone information and facility configuration information from the data in gbXML file format and loads it into the system (1). gbXML is a file format used to transmit 3D CAD data as a tool for energy analysis, and is a type of language that enables 3D BIM and architecture / engineering analysis software to share data with each other.

[0056] At this time, the above space information may be a single or multiple selected from space ID, name, area, estimated number of personnel, and equipment load, and the above equipment configuration information may be a single or multiple selected from equipment ID, equipment type (HVAC, lighting, power, etc.), list of equipment by space, list of controllable equipment, location coordinates, and connection zone.

[0057] The above HMI module (200) is an HMI, which stands for Human Machine Interface, and is a user interface for interaction between a user and a system, and includes a dashboard or display screen used for system control, and is equipped to monitor the operating status of various facilities of a building through graphics and to take appropriate measures when necessary.

[0058] The above HMI module (200) receives a user's query regarding a control request in the form of natural language, transmits it to the AI ​​module (300), and requests verification of whether the control currently being performed is an optimized control. The user's query can be input into the HMI module (200) in voice or text, and the natural language query can be input into the HMI module (200) in voice or text through the administrator module (400).

[0059] For example, when a user queries the HMI module (200) with a control request to “make it brighter” through the administrator module (400), the AI ​​module (300) interprets the query and generates a control scenario based on prior knowledge or feedback, and provides a control recommendation to the user through the HMI module (200) to make it 20% brighter, “illumination 70 lux,” and if approved, the HMI module (200) generates a control signal and transmits the control command for “illumination 70 lux” to the corresponding lighting equipment so that optimized control execution can be performed.

[0060] The above HMI module (200) distinguishes a list of controllable equipment according to the authority level of the user account, and some controls may require administrator privileges.

[0061] The above HMI module (200) transmits environmental status information and control history of the building, periodically collected based on real-time sensor data, to the AI ​​module (300). The above HMI module (200) periodically collects environmental status information, such as the building's temperature, humidity, illuminance, and occupancy count, based on real-time sensor data and transmits it to the AI ​​module (300) in JSON format. JSON (JavaScript Object Notation) is a character-based standard format for expressing structured data using JavaScript object syntax.

[0062] When the above HMI module (200) receives a confirmed control plan from the AI ​​module (300), it generates a control signal and transmits a control command to the corresponding equipment so that optimized control execution is performed.

[0063] The above HMI module (200) can receive user feedback based on the control execution result in voice or text and transmit it to the feedback learning module (500) to be described later.

[0064] The AI ​​module (300) receives building environmental status information and control history, logs and analyzes them as training data, interprets user queries based on a large-scale language model (LLM), integrates and analyzes the interpreted queries with the spatial information and environmental status information to infer control objectives and generate control scenarios, and provides control recommendations to the user through the HMI module (200). It also receives user approval or instructions for supplementation regarding the control recommendations through the HMI module (200), supplements the control recommendations, and transmits the approved or supplemented finalized control plans to the HMI module. Furthermore, when generating control scenarios, the AI ​​module (300) can generate differentiated control scenarios for the same user control request by considering external environmental conditions such as time of day, day of the week, and external weather.

[0065] More specifically, referring to FIG. 5b, the HMI module (200) may comprise a state analysis unit (310), a natural language query interpretation unit (320), a goal inference unit (330), a scenario generation unit (340), and a control execution unit (350).

[0066] The above state analysis unit (310) receives environmental state information and control history, such as the temperature, humidity, illuminance, and occupancy of the building, based on real-time sensor data, logs and analyzes it as learning data.

[0067] The above natural language query interpretation unit (320) interprets the user's query based on a large language model (LLM). An LLM (Large Language Model) is a model capable of understanding and generating natural language by learning large-scale text data. The above natural language query interpretation unit (320) can identify the user's question intent based on prior learned knowledge, and the precision of interpreting the natural language query is improved based on the user's subjective purpose.

[0068] The above objective inference unit (330) first automatically infers the control objective by integrating and analyzing the query interpreted by the natural language query interpretation unit (320), spatial information, and environmental state information.

[0069] The above scenario generation unit (340) generates a control scenario and provides a control recommendation to the user, and generates the control scenario by integrating and analyzing the query interpreted by the natural language query interpretation unit (320), spatial information, and environmental status information. At this time, the control recommendation is output in JSON format and may include a space name, identifier, control facility ID, control command, target value, reason for judgment, and judgment reliability.

[0070] For example, when a user’s query is entered as “cool the conference room” in natural language through the HMI module (200), the purpose inference unit (330) can analyze the interpretation of the query, spatial information about the conference room, and information about the environmental condition of the conference room to infer a control objective that cooling is required, and apply a predetermined judgment criterion under the inferred control objective to create a control scenario for cooling by setting the target temperature of the conference room to 22.5℃ and providing this to the user as a control recommendation, and the control execution result can be fed back and reflected in future judgments.

[0071] The control execution unit (350) receives the user's approval or supplementary instructions regarding the control recommendation, supplements the control recommendation if supplementary instructions are given, and transmits the approved or supplemented final control confirmation plan to the HMI module (200) so that optimal control according to the user's intention is executed, thereby enabling the automatic generation of scenarios according to the user's subjective purpose, allowing the system to be operated efficiently, saving energy, and improving the user's comfort.

[0072] The above-mentioned operator module (400) is a user corresponding to a building manager or control system operator who checks the control recommendations provided by the AI ​​module (300) and approves or provides supplementary instructions regarding the control recommendations to the HMI module (200). At this time, the operator module (400) may be any one of a desktop PC, laptop, smartphone, tablet PC, smartwatch, or PDA, and may be connected to the system (1) wirelessly or via a wired connection so that natural language queries can be input via voice or text.

[0073] If the user approves the control recommendation, the control recommendation becomes the final confirmed control plan; if the user issues instructions to supplement the control recommendation, the plan supplemented by the AI ​​module (300) becomes the final confirmed control plan, and the results of such control execution can be fed back and reflected in the next control. If the control methods of the BAS and the AI ​​module (300) differ, the user makes the final decision and instructs supplementation if necessary. In addition, the user queries the AI ​​module (300) for cost and efficiency information, and if the AI ​​module (300) produces results different from expectations, it analyzes the cause.

[0074] For example, as in the example described above, a control scenario is created to lower the target temperature of the meeting room to 22.5℃ and provided to the user as a control recommendation. However, if the user inputs a supplementary instruction to "make it a little cooler" through the HMI module (200), the AI ​​module (300) executes a supplementary control confirmation plan by adjusting the target temperature of the meeting room to 21.5℃, and the result of this control execution is fed back so that it can be reflected in the next control when the same condition occurs in the future.

[0075] In this way, an integrated platform configuration including an HMI-based integrated control interface, real-time monitoring, control recommendations of the AI ​​module (300), and user feedback reflection is created, and an optimal control scenario is generated by the judgment of the AI ​​module (300) and the decision of the user, enabling real-time optimized control of energy and efficient operation, and improving user comfort and satisfaction.

[0076] The feedback learning module (500) generates an Interaction ID for feedback identification by combining user queries and environment status information, and learns by collecting and storing scenario generation results from the AI ​​module (300), control execution results from the HMI module (200), and user feedback based on the Interaction ID. When the AI ​​module (300) generates a control scenario, it searches for feedback data for the same conditions through the feedback learning module (500). If no feedback exists, it generates a scenario based on the prior-learned knowledge of the LLM. If feedback exists, it generates a feedback-based control scenario by applying a corrected judgment criterion that reflects the searched feedback data. At this time, the feedback learning module (500) stores and manages scenario generation results, control execution results, and user feedback based on the Interaction ID, and when such feedback data accumulates, the judgment criterion for each condition is automatically corrected.

[0077] In this way, it has an interconnected cyclical structure of BIM-HMI-AI-User (Operator) and provides a cyclical structure that learns user feedback to enhance control scenarios. Conditional judgment criteria are automatically corrected based on accumulated feedback, and judgment criteria are refined through self-learning via feedback. Optimal control is achieved to align with the user's intent, enabling the system to operate efficiently and improve user comfort and satisfaction while saving energy.

[0078] The above API module (600) provides an API (Application Programming Interface) that can be linked with external systems such as a building energy management system (BEMS) or a factory energy management system (FEMS), and can perform integrated control by exchanging real-time data with said external systems, and is easy to expand.

[0079] Ultimately, the building energy control method based on natural language processing LLM-AI according to the present invention is applicable to various buildings such as smart buildings, hospitals, schools, and public institutions, and enables control of various targets including HVAC, lighting, and power. It is also applicable to edge computing and cloud-based control systems. By interpreting a user's natural language query based on a Large Language Model (LLM) and integrating and analyzing it with building environmental status information periodically collected based on real-time sensor data and building information including spatial and facility configurations received from a Building Information Model (BIM), it infers control objectives, generates control scenarios, and provides control recommendations to execute energy optimization control for facilities. Moving away from conventional fixed, static logic-based control methods, it generates AI-based control scenarios according to the user's purpose and provides control recommendations, thereby improving the accuracy and consistency of energy control, saving energy, reducing user judgment fatigue, preventing errors, enhancing user comfort and satisfaction, and improving responsiveness to anomaly detection. Furthermore, through a cyclic learning structure of natural language query interpretation, integrated analysis, and user feedback learning and application, it enables real-time energy optimization control and efficient operation. Finally, the AI The module performs integrated analysis to infer control objectives, and through repeated user feedback learning, its judgment criteria self-learn and become more sophisticated, gradually optimizing to meet user intent. It can be linked in a modular form with external systems such as BEMS / FEMS, is easy to introduce and expand, and enables integrated control.

[0080] The above embodiments in the present invention are merely examples, and the present invention is not limited thereto. Any configuration having substantially the same structure as the technical concept described in the claims of the present invention and achieving the same functional effect is included within the technical scope of the present invention. Explanation of the symbols

[0082] 1. System 100. Information Collection Module 200. HMI Module 300. AI Module 310. State Analysis Department 320. Natural Language Query Parsing Unit 330. Purpose Inference Unit 340. Scenario Generation Section 350. Control Execution Unit 400. Administrator Module 500. Feedback Learning Module 600. API Module B. BEMS F. FEMS

Claims

Claim 1 The present invention relates to a method for real-time energy control through the interaction of a Human Machine Interface (HMI), AI, and a user utilizing Building Information Model (BIM)-based information, comprising the steps of: an information collection module receiving spatial information and equipment configuration information from the Building Information Model (BIM) and transmitting it to an AI module; an HMI module periodically transmitting building environmental status information and control history collected based on real-time sensor data to an AI module in JSON (JavaScript Object Notation) format, and the AI ​​module logging and analyzing the building environmental status information and control history as training data; an HMI module receiving a user query regarding a control request in natural language form and transmitting it to an AI module; an AI module interpreting the user query based on a Large Language Model (LLM), integrating and analyzing the interpreted query with the spatial information and environmental status information to infer the control objective, generating a control scenario, and providing a control recommendation to the user; and a user reviewing the control recommendation provided by the AI ​​module and approving or instructing the HMI module to supplement the control recommendation. A step in which the AI ​​module receives user approval or supplementary instructions regarding a control recommendation, transmits the finalized control plan approved by the user to the HMI module, or receives the user's supplementary instructions to supplement the control recommendation and provides the supplemented finalized control plan to the HMI module; a step in which the HMI module receives the finalized control plan, generates a control signal, and transmits a control command to the relevant equipment so that control execution is performed;A method for controlling building energy by a natural language processing-based LLM-AI, comprising the step of an HMI module transmitting control execution results and user feedback to a feedback learning module, wherein the feedback learning module generates an Interaction ID for feedback identification by combining user queries and environmental state information, and further comprises the step of collecting and storing scenario generation results, control execution results, and user feedback based on the Interaction ID for learning, wherein the AI ​​module searches for feedback data for the same conditions through the feedback learning module when generating a control scenario, generates a scenario based on prior learned knowledge of the LLM if feedback data does not exist, and generates a feedback-based control scenario by applying judgment criteria corrected by the searched feedback data if feedback data exists, and performs real-time control of energy through a cyclic learning structure of natural language query interpretation, integrated analysis, and user feedback learning and application. Claim 2 A building energy control method based on natural language processing LLM-AI according to claim 1, wherein the information regarding the space is a single or multiple selected from space ID, name, area, estimated number of personnel, and equipment load, and the information regarding the equipment configuration is a single or multiple selected from equipment ID, type of equipment, list of equipment by space, list of controllable equipment, location coordinates, and connection zone, and the HMI module distinguishes the list of controllable equipment according to the authority level of the user account, and some controls require administrator authority. Claim 3 In claim 1, the control recommendation is output in JSON (JavaScript Object Notation) format, and the building energy control method by natural language processing-based LLM-AI includes a space name, identifier, control facility ID, control command, target value, reason for judgment, and judgment reliability. Claim 4 A building energy control method by natural language processing-based LLM-AI according to claim 1, characterized in that the AI ​​module generates differential control scenarios for a user's query regarding the same control request by considering external environmental conditions such as time of day, day of the week, and external weather when generating control scenarios. Claim 5 A building energy control method based on natural language processing LLM-AI, characterized in that, in claim 1, an Application Programming Interface (API) module capable of interoperability with external systems such as a building energy management system (BEMS) or a factory energy management system (FEMS) can perform integrated control by exchanging real-time data with said external systems. Claim 6 delete

Images