Building automation system, control method, device, equipment, medium and product

Abstract

The embodiment of the invention discloses a building automation system, a control method, a device, equipment, a medium and a product. The system comprises a database used for storing equipment data of a building; the intelligent agent is used for receiving natural language statements related to the building; the large language model is used for executing semantic understanding on the natural language statement and generating an operation task based on a semantic understanding result; wherein the intelligent agent is also used for accessing the database to obtain the equipment data associated with the operation task from the equipment data of the building, and executing the operation task based on the equipment data associated with the operation task. And the large language model is cooperated with the intelligent agent, so that the building task is automatically completed, and the intelligence of building control is improved.

Description

Technical Field

[0001] The present invention relates to the field of automatic control technology, in particular to building automation systems, control methods, devices, equipment, media and products. Background Art

[0002] Building control, also known as building automation or building automation, refers to the process of using specialized systems to monitor, control, and manage the various devices and functions within a building. Building control systems can improve energy efficiency, safety, comfort, and convenience, while also reducing operating costs.

[0003] The intelligence of building automation is to achieve automated and intelligent management and control of various equipment and systems in buildings through advanced technical means, so as to improve energy efficiency, enhance comfort, optimize management efficiency, and enhance system reliability and flexibility.

[0004] At present, how to improve the intelligence level of building automation is one of the focuses of the industry. Summary of the Invention

[0005] The embodiments of the present invention provide a building automation system, a control method, an apparatus, equipment, a medium and a product.

[0006] A building automation system comprising:

[0007] Database, used to store building equipment data;

[0008] An intelligent agent, configured to receive a natural language sentence about the building;

[0009] A large language model is used to perform semantic understanding on the natural language sentence and generate an operation task based on the semantic understanding result;

[0010] The intelligent agent is further configured to access the database to obtain device data associated with the operation task from the device data of the building, and execute the operation task based on the device data associated with the operation task.

[0011] Therefore, the embodiments of the present invention coordinate the large language model with the intelligent agent, wherein the large language model performs natural language processing on the natural language received by the intelligent agent and controls the intelligent agent to access the database containing device data. It can automatically complete various building tasks (such as deep operation and maintenance and energy management, etc.) in a natural language manner, significantly improving the intelligence of building control.

[0012] In one embodiment, the agent includes a pre-set prompt word template;

[0013] The intelligent agent is configured to, when the natural language sentence conforms to the prompt word template, send the natural language sentence to the large language model so that the large language model performs semantic understanding on the natural language sentence; and when the natural language sentence does not conform to the prompt word template, filter the natural language sentence;

[0014] The prompt word template includes at least one of the following: a location field; a device field; a parameter field; a command field; and a keyword field.

[0015] It can be seen that by setting the prompt word template in the intelligent agent, the frequent call of the large language model can be effectively prevented, thereby improving the processing speed and control efficiency.

[0016] In one embodiment, the semantic understanding result includes a position and an output type;

[0017] The operation task includes at least one of the following:

[0018] Reading device data corresponding to the location and associated with the output type from the database;

[0019] determining output data that matches the output type based on the device data associated with the output type that matches the location;

[0020] The output data is output based on a human-computer interaction interface.

[0021] It can be seen that based on the semantic understanding results including location and output type, the operation tasks can be automatically generated, thereby improving the degree of automation of building control.

[0022] In one embodiment, the semantic understanding result includes the position and the target value of the controlled parameter;

[0023] The operation task includes at least one of the following:

[0024] reading a current value of the controlled parameter at the location from the database;

[0025] determining whether to adjust the current value to the target value based on the knowledge in the local database of the large language model;

[0026] When it is determined that the current value is not adjusted to the target value, an error prompt is issued based on the human-computer interaction interface;

[0027] When it is determined that the current value is adjusted to the target value, the target value is sent to a configuration field of the controlled device at the location in the database for controlling the controlled parameter.

[0028] It can be seen that based on the semantic understanding results of the target values including the position and the controlled parameters, the operation tasks can be automatically generated, thereby improving the degree of automation of building control.

[0029] In one embodiment, the semantic understanding result includes a location and predetermined keywords;

[0030] The operation task includes at least one of the following:

[0031] Reading from the database the current value of the controlled parameter associated with the keyword that matches the location;

[0032] determining an adjustment value for the current value based on the knowledge in the local database of the large language model and the keyword;

[0033] The adjustment value is sent to a configuration field of a controlled device at the location in the database for controlling the controlled parameter.

[0034] It can be seen that based on the semantic understanding results including location and keywords, operation tasks can be automatically generated, thereby improving the degree of automation of building control.

[0035] In one embodiment, the semantic understanding result includes the location, type and action of the controlled device;

[0036] The operation task includes at least one of the following:

[0037] Reading from the database the current status of the controlled device at the location that meets the type;

[0038] determining an adjustment state of the current state based on the knowledge in the local database of the large language model and the action;

[0039] The adjustment status is sent to a configuration field of a controlled device of the type at the location in the database.

[0040] It can be seen that based on the semantic understanding results including location, type of controlled equipment and action, operation tasks can be automatically generated, thereby improving the degree of automation of building control.

[0041] In one embodiment, the database is a Haystack database;

[0042] The large language model is used to generate an operation task including a label based on the semantic understanding result;

[0043] The intelligent agent is configured to access the Haystack database based on the tag to obtain device data associated with the operation task;

[0044] wherein the tag comprises at least one of the following:

[0045] Site tag; floor tag; area tag; equipment tag; temperature tag; humidity tag; carbon dioxide tag; occupancy tag.

[0046] Therefore, the intelligent agent accesses the Haystack database through labels to obtain relevant data, without the need to directly connect the large language model to the building equipment. This solves the defect that the large language model has difficulty in obtaining device data, improves the degree of automation, and reduces the difficulty of connection.

[0047] In one embodiment, the number of tags of the operation task is multiple;

[0048] The intelligent agent is configured to access the Haystack database one by one using each tag of the operation task in a predetermined tag sequence to obtain the intermediate device data obtained by each access; and determine the device data associated with the operation task based on the intermediate device data obtained by each access; or

[0049] The intelligent agent is used to combine multiple tags of the operation task into a tag chain according to a predetermined tag sequence; and use the tag chain to access the Haystack database to obtain device data associated with the operation task.

[0050] It can be seen that device data associated with operation tasks can be obtained through multiple tag access methods, which improves convenience.

[0051] In one embodiment, the large language model is used to generate the operation task based on the semantic understanding result in a thinking chain based on a linear thinking process or a thinking tree based on a branching structure.

[0052] Therefore, the operation tasks are generated in the form of thought chains or thought trees. Thought chains can help the automation system understand instructions more clearly and generate operation tasks, while thought trees can select the optimal path to generate operation tasks.

[0053] A building control method, comprising:

[0054] receiving natural language sentences about the building;

[0055] inputting the natural language sentence into a large language model, wherein the large language model performs semantic understanding on the natural language sentence to generate an operation task based on a semantic understanding result;

[0056] Accessing a database storing equipment data of the building to obtain equipment data associated with the operation task;

[0057] The operation task is performed based on the device data associated with the operation task.

[0058] Therefore, the embodiments of the present invention coordinate the large language model with the intelligent agent, wherein the large language model performs natural language processing on the natural language received by the intelligent agent and controls the intelligent agent to access the database containing device data. It can automatically complete various building tasks (such as deep operation and maintenance and energy management, etc.) in a natural language manner, significantly improving the intelligence of building control.

[0059] In one embodiment, before inputting the natural language sentence into the large language model, the method includes:

[0060] determining whether the natural language sentence conforms to a predetermined prompt word template, wherein when the natural language sentence conforms to the prompt word template, sending the natural language sentence to the large language model so that the large language model performs semantic understanding on the natural language sentence; and when the natural language sentence does not conform to the prompt word template, filtering the natural language sentence;

[0061] The prompt word template includes at least one of the following: a location field; a device field; a parameter field; a command field; and a keyword field.

[0062] It can be seen that by setting the prompt word template, the frequent calling of the large language model can be effectively prevented, thereby improving the processing speed and control efficiency.

[0063] In one embodiment, the database is a Haystack database;

[0064] The generating of the operation task based on the semantic understanding result includes: generating the operation task including the label based on the semantic understanding result;

[0065] The accessing a database for storing equipment data of a building to obtain equipment data associated with the operation task includes: accessing the Haystack database based on the tag to obtain equipment data associated with the operation task;

[0066] wherein the tag comprises at least one of the following:

[0067] Site tag; floor tag; area tag; equipment tag; temperature tag; humidity tag; carbon dioxide tag; occupancy tag.

[0068] Therefore, by accessing the Haystack database through tags to obtain relevant data, there is no need to directly connect the large language model to the building equipment. This solves the defect that large language models have difficulty in obtaining device data, improves the degree of automation, and reduces the difficulty of connection.

[0069] In one embodiment, the number of tags of the operation task is multiple;

[0070] Accessing the Haystack database based on the tag to obtain device data associated with the operation task includes:

[0071] Accessing the Haystack database one by one using each tag of the operation task in a predetermined tag sequence to obtain the intermediate device data obtained in each access; determining the device data associated with the operation task based on the intermediate device data obtained in each access; or

[0072] According to a predetermined tag sequence, multiple tags of the operation task are combined into a tag chain; and the tag chain is used to access the Haystack database to obtain device data associated with the operation task.

[0073] Therefore, device data associated with operation tasks can be obtained through multiple tag access methods, which improves convenience.

[0074] A building control device, comprising:

[0075] A receiving module, configured to receive natural language sentences about buildings;

[0076] an input module, configured to input the natural language sentence into a large language model, wherein the large language model performs semantic understanding on the natural language sentence to generate an operation task based on the semantic understanding result;

[0077] An access module, configured to access a database storing equipment data of the building to obtain equipment data associated with the operation task;

[0078] An execution module is configured to execute the operation task based on the device data associated with the operation task.

[0079] An electronic device comprising a processor and a memory;

[0080] The memory stores an application program executable by the processor, which is used to enable the processor to execute any of the above-mentioned building control methods.

[0081] A computer-readable storage medium stores computer instructions, wherein the computer instructions, when executed by a processor, implement any of the above-mentioned building control methods.

[0082] A computer program product comprises a computer program, wherein when the computer program is executed by a processor, the computer program implements any one of the above building control methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0083] The preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, so that those skilled in the art will understand the above and other features and advantages of the present invention more clearly. In the accompanying drawings:

[0084] Figure 1 is a structural diagram of a building automation system according to an embodiment of the present invention.

[0085] Figure 2 is a first exemplary schematic diagram of a building automation system according to an embodiment of the present invention.

[0086] Figure 3 is a second exemplary schematic diagram of a building automation system according to an embodiment of the present invention.

[0087] Figure 4 2 is a schematic diagram of generating operation tasks in a mind tree manner according to an embodiment of the present invention.

[0088] Figure 5 2 is a schematic diagram of generating an operation task in a thought chain manner according to an embodiment of the present invention.

[0089] Figure 6 is an exemplary flow chart of a building control method according to an embodiment of the present invention.

[0090] Figure 7 is an exemplary structural diagram of a building control device according to an embodiment of the present invention.

[0091] Figure 8 is an exemplary structural diagram of an electronic device according to an embodiment of the present invention.

[0092] The accompanying drawings are numerals as follows:

[0093]

[0094] DETAILED DESCRIPTION

[0095] To make the purpose, technical solutions and advantages of the present invention more clear, the present invention is further described in detail with reference to the following embodiments. Nouns and pronouns related to people in this patent application are not limited to specific genders.

[0096] For the sake of brevity and intuitiveness in description, the solution of the present invention is explained below by describing several representative implementations. A large number of details in the implementations are only used to help understand the solution of the present invention. However, it is obvious that the technical solution of the present invention may not be limited to these details when implemented. In order to avoid unnecessarily obscuring the solution of the present invention, some implementations are not described in detail, but only a framework is given. Hereinafter, "including" means "including but not limited to", and "according to..." means "at least according to..., but not limited to only according to...". Due to the language habits of Chinese, when the number of a component is not specifically specified below, it means that the component can be one or more, or can be understood as at least one.

[0097] Large language models (LLMs) generally refer to models with a large number of parameters in the field of artificial intelligence, especially in machine learning and deep learning. Because of their numerous parameters, large language models are able to capture and learn complex patterns and relationships in data, thereby demonstrating excellent performance on a variety of tasks. An agent is a key concept in the field of artificial intelligence, referring to an entity that can autonomously perform tasks or make decisions in a specific environment. An agent can be a software program, robot, or other form of system that can perceive its environment and take autonomous actions to achieve a specific goal.

[0098] The applicant found that if a large language model can be introduced into intelligent building control, the intelligence of the building control process can be significantly improved. However, directly introducing a large language model into intelligent building control is difficult to operate. This is because: the large language model is difficult to simply connect with building equipment (usually requires the use of complex communication protocols or interface calls), and it is especially difficult to directly access the database containing equipment data (such as the Haystack database), which makes it difficult to implement the large language model to directly control the building. The applicant further found that: if an intelligent agent that can access (for example, through tags) a database containing equipment data is provided, the large language model and the intelligent agent are coordinated with each other, wherein the large language model performs natural language processing on the natural language received by the intelligent agent and controls the intelligent agent to access the database containing equipment data, then the large language model can automatically complete various building tasks (such as deep operation and maintenance and energy management, etc.) in a natural language manner, thereby significantly improving the intelligence of building control.

[0099] The above disclosure details the technical defects in the related art, the causes of these defects, and the analytical process for overcoming them. In reality, the understanding of these technical defects is not common knowledge in the field, but rather a novel discovery made by the inventors during their research. Furthermore, the tracing of the causes of these defects and the analytical process for overcoming them are the result of gradual analysis conducted by the inventors during their actual research and are not common knowledge in the field.

[0100] Figure 1 FIG is a structural diagram of a building automation system according to an embodiment of the present invention. Figure 1 As shown, the building automation system includes: a database 13 for storing equipment data of the building; an intelligent agent 12 for receiving natural language statements about the building; a large language model 11 for performing semantic understanding of the natural language statements and generating operation tasks based on the semantic understanding results; wherein the intelligent agent 12 is also used to access the database 13 to obtain equipment data associated with the operation task from the equipment data of the building, and perform the operation task based on the equipment data associated with the operation task.

[0101] Equipment 14 in a building refers to equipment that can be monitored and controlled through an automation system. Equipment 14 can cover all aspects of a building to ensure efficient operation, energy conservation, and a comfortable environment. For example, equipment 14 may include: (1) Heating, ventilation, and air conditioning system (HVAC): air conditioning units; cold and heat source equipment (such as boilers, chillers); fan coil units; fresh air systems; ventilation equipment. (2) Lighting system: indoor lighting (including LED lights, fluorescent lights, etc.); outdoor lighting (such as street lights, landscape lights); emergency lighting. (3) Elevators and escalators: vertical elevators; escalators. (4) Security system: video surveillance cameras; access control systems (including card readers, biometric devices); alarm systems (such as fire alarms, intrusion alarms). (5) Water supply and drainage system: water pumps; valves; rainwater drainage systems. (6) Energy management system: electricity meters; energy monitoring equipment. (7) Environmental monitoring equipment: temperature sensors; humidity sensors; air quality sensors (such as CO2, PM2.5 monitoring). (8) Curtain and sunshade system: automatic curtains; sunshades. (9) Audio and public address system: background music playback; emergency broadcast;

[0102] (10) Parking lot management system: vehicle detector; gate control; etc.

[0103] The database 13 plays a key role in intelligent building management systems. It is used to store and manage data related to various devices 14 within the building (i.e., device data). Device data is crucial for building automation, energy management, device maintenance, and fault diagnosis. For example, device data may include:

[0104] (1) Sensor data

[0105] Sensor data is typically collected in real time by various sensors. For example, sensor data may include, but is not limited to: Temperature and humidity: used to monitor the indoor environment and adjust air conditioning and ventilation systems. Light intensity: used to control lighting systems and achieve intelligent adjustment of natural and artificial light. Energy consumption: such as meter readings, used to monitor and optimize energy use. Security monitoring: such as images captured by cameras, used for security monitoring and intrusion detection. Environmental quality: such as carbon dioxide concentration and PM2.5, used to assess indoor air quality; and so on.

[0106] (2) Intermediate data

[0107] Intermediate data is generated during data processing and is typically used for further analysis or as input to other systems. Intermediate data may include, but is not limited to: Statistical data, such as daily average energy consumption and monthly average temperature, used for energy management and environmental control; Trend analysis, such as energy consumption trends and equipment usage frequency, used for forecasting and optimization; Alarm and event logging, which records equipment failures and security incidents for maintenance and safety management.

[0108] (3) Configuration data

[0109] Configuration data refers to the settings and parameters of building equipment, used to control and adjust their behavior. Configuration data may include, but is not limited to: Equipment parameters, such as air conditioning temperature settings and lighting system brightness adjustments; System configurations, such as automation control logic and security system response strategies; and User preferences, such as user-specific temperature or lighting settings.

[0110] In one embodiment, the database 13 can be implemented as a database stored based on the Haystack data model (referred to as a Haystack database). Haystack is an open source metadata storage format designed to store time series data and other related metadata that can be used to describe the attributes of data points. In smart building systems, Haystack can be used to effectively organize and query device data, such as sensor data, intermediate data, and configuration data. When using Haystack to store device data, setting tags is a key step. Tags can help organize data, improve query efficiency, and enhance data readability. For example, the specific process of setting tags includes:

[0111] (1) Define the tag structure:

[0112] Determine what tags are needed to describe the device data. For example, you might need tags to describe the device type, location, manufacturer, installation date, etc.

[0113] (2) Create a tag:

[0114] In Haystack, tags are usually stored in the form of key-value pairs. For example, you can create the following tags for the temperature sensor:

[0115]

[0116] Among them: "location" is the location tag; "manufacturer" is the manufacturer tag; "installation_date" is the installation date tag; "device_type" is the device type tag.

[0117] (3) Application tags:

[0118] Apply defined labels to corresponding data points. In Haystack, this involves attaching these labels as data is written, ensuring that each data point has enough labels to be accurately described and queried.

[0119] (4) Storage and indexing:

[0120] Store tagged data in a Haystack-compatible database. Haystack supports a variety of databases, such as InfluxDB and SQL databases. Make sure to index by tags to speed up queries.

[0121] (5) Query and analysis:

[0122] Use Haystack's query language or API to retrieve data with specific tags. For example, you can query for data about all devices located at "building_1_floor_3." Tags can be used for data analysis, such as calculating the energy consumption of specific types of equipment or monitoring environmental parameters in a specific area.

[0123] (6) Maintenance and Update:

[0124] Regularly review and update tags to ensure they reflect the latest equipment status and configuration. If equipment is moved or reconfigured, update the corresponding tags to maintain data accuracy.

[0125] For example, assume that a smart building system contains multiple sensors. A unique identifier can be created for each sensor and a set of tags can be assigned to it.

[0126]

[0127] In the example above, sensor_123 is a temperature sensor, and its tags describe the device type (device_type), location (location), manufacturer (manufacturer), and installation date (installation_date). These tags help quickly query and analyze data from specific devices. By setting tags in this way, Haystack can effectively organize and manage building device data, thereby improving data accessibility and analysis capabilities.

[0128] With the development of artificial intelligence technology, multimodal learning is becoming increasingly popular. Multimodal learning refers to the ability of a model to simultaneously process and understand multiple types of data, such as text, images, audio, and video.

[0129] In one embodiment, the intelligent agent 12 can receive various types of natural language sentences about buildings from the user 10. For example, natural language sentences can be implemented as text, images, audio, and video. In order for the intelligent agent to process non-text data, it is generally necessary to convert this data into a format that the intelligent agent can understand. For example, features can be extracted from images and videos using convolutional neural networks (CNNs), and features can be extracted from audio using spectrograms or Mel-Frequency Cepstral Coefficients (MFCCs). These features, along with the text data, are then input into a large language model.

[0130] The large language model 11 performs semantic understanding on natural language sentences and generates operational tasks based on the semantic understanding results. The specific process includes: the large language model 11 receives natural language sentences; the large language model 11 converts the natural language sentences into text; the large language model 11 cleans and standardizes the text (such as word segmentation, stop word removal, stemming, etc.); the large language model 11 performs semantic understanding on the cleaned and standardized text (such as word embedding, context understanding, syntactic and semantic parsing, intent recognition, entity recognition, etc.); and based on the semantic understanding results, the large language model 11 determines the operational tasks to be performed.

[0131] In one embodiment, the large language model 11 can be implemented as: a Transformer-based model (such as BERT, GPT, or DeepSeek, etc.), a multimodal model, a RoBERTa model, a T5 model, etc.

[0132] For example: User 10 sends a natural language instruction to the agent 12 through a smart speaker, input text box or mobile device: "Please adjust the temperature of the lobby on the first floor to 22 degrees Celsius and open the curtains on the west side at 5 pm." The agent 12 sends the instruction to the large language model 11. The large language model 11 recognizes that the user's intention is to adjust the temperature and the curtain status, and recognizes specific entities such as "lobby on the first floor" and "curtains on the west side." Based on the recognized intentions and entities, the large language model 11 defines two tasks: adjusting the temperature and adjusting the curtains. For the temperature adjustment task, the corresponding operation task (that is, action sequence) is generated: query the current temperature->send an adjustment instruction to the HVAC system->confirm that the temperature has been adjusted. For the curtain adjustment task, the corresponding operation task (that is, action sequence) is generated: query the current curtain status->send an open instruction to the curtain control system->confirm that the curtains are open.

[0133] To prevent the agent from ineffectively and frequently calling the large language model, a prompt word template can be set to allow the agent to filter out irrelevant natural language sentences.

[0134] In one embodiment, the intelligent agent 12 includes a pre-set prompt word template; the intelligent agent 12 is used to send the natural language sentence to the large language model 11 when the natural language sentence meets the prompt word template so that the large language model 11 performs semantic understanding on the natural language sentence; when the natural language sentence does not meet the prompt word template, filter the natural language sentence (for example, the natural language sentence is not sent to the large language model 11); wherein the prompt word template includes at least one of the following: location field; device field; parameter field; command field; keyword field.

[0135] For example: Assume that the prompt word template contains a position field, a command field, and a parameter field.

[0136] (1) When the natural language sentence is "Please query the temperature data of room 407," agent 12 parses the natural language sentence and finds the query contains the location field ("room 407"), the command field ("query"), and the parameter field ("temperature"), thus determining that the natural language sentence meets the prompt word template. Agent 12 then sends the natural language sentence to large language model 11, which performs semantic understanding on the natural language sentence.

[0137] (2) When the natural language sentence is "What are the factors that affect room temperature?", agent 12 parses the natural language sentence and finds that it cannot find the location field, command field, and parameter field. Agent 12 then determines that the natural language sentence does not match the prompt word template. In this case, agent 12 does not send the natural language sentence to the large language model 11, but instead filters the natural language sentence.

[0138] It can be seen that by setting the prompt word template in the intelligent agent, the frequent call of the large language model can be effectively prevented, thereby improving the processing speed and control efficiency.

[0139] This can be implemented in a variety of ways, such as Figure 1 Building automation system shown.

[0140] Figure 2 is a first exemplary schematic diagram of a building automation system according to an embodiment of the present invention. Figure 2 In the embodiment, the agent 12 and the large language model 11 are integrated with each other to form an artificial intelligence device 20 as a whole. In the artificial intelligence device 20, the agent 12 calls the large language model 11 based on the local interface.

[0141] Figure 3 is a second exemplary schematic diagram of a building automation system according to an embodiment of the present invention. Figure 3 In the embodiment, the large language model 11 is arranged in the cloud 30. The agent 12 accesses the cloud 30 based on a remote interface to call the large language model 11 in the cloud 30.

[0142] In one embodiment, the large language model 11 generates operation tasks based on the semantic understanding results by simulating the chain of thought (CoT) based on the linear thinking process. The chain of thought method can help the model more clearly display its decision-making process, thereby improving the transparency and explainability of the operation. In one embodiment, the large language model 11 generates operation tasks in the form of a tree of thoughts (ToT) based on a branch structure based on the semantic understanding results. The thinking tree provides a clear decision path, showing the entire process from problem to solution, which helps to understand and track the decision-making process.

[0143] Figure 4 This is a schematic diagram of generating operational tasks using a mindset tree approach according to an embodiment of the present invention. The mindset tree simulates the human thought process when faced with complex problems, exploring different solution paths by constructing a decision tree and selecting the optimal path to generate the operational task.

[0144] For example, the processing process of the large language model 11 includes: (1) Input processing: receiving natural language sentences and performing preprocessing, such as word segmentation and removal of stop words. (2) Semantic understanding: performing semantic analysis on natural language sentences to identify key intentions, entities, and relationships. (3) Constructing a thinking tree: based on the results of semantic understanding, construct an initial thinking tree, where each node represents a possible operation or decision point. The root node of the tree represents the initial state or problem, and the leaf nodes represent possible solutions or operation tasks. (4) Branch expansion: starting from the root node, multiple branches are expanded based on possible operations or decisions. Each branch represents a possible action path, and the model will evaluate the feasibility and effectiveness of each path. (5) Evaluation and selection: evaluating the leaf nodes of each branch, considering factors such as efficiency, cost, and security. Select the optimal branch as the final operation task. (6) Task execution: generating specific operation tasks based on the selected optimal path.

[0145] exist Figure 4 In the representation task definition, root node 50 includes branches 51, 52, and 53. Branch 51 includes sub-branch 61. Branch 52 includes sub-branches 63-65. Branch 53 includes sub-branch 66. Sub-branch 61 includes grandchildren 62 and 68. Sub-branch 63 includes grandchildren 68. Sub-branch 64 includes grandchildren 69. Sub-branch 65 includes grandchildren 69, 70, and 71. Sub-branch 66 includes grandchildren 71. All branches ultimately converge at node 72, indicating the completion of the representation task.

[0146] For example, suppose user 10 issues the following instruction to agent 12: "I have a video conference tomorrow morning at 9:00 AM. Please ensure the room temperature, lighting, and video equipment are working properly." Agent 12 sends this instruction to large language model 11. The semantic understanding output from large language model 11 includes the following: intent: prepare the meeting room; entities: meeting room, temperature, lighting, video equipment; time: tomorrow morning at 9:00 AM.

[0147] The thinking tree constructed by the large language model 11 includes: a root node: prepare a meeting room; and branches 1, 2, and 3 below the root node.

[0148] Branch 1: Adjust the temperature, wherein branch 1 also includes: sub-branch 1.1: query the current temperature; sub-branch 1.2: set the target temperature (for example, 22°C); sub-branch 1.3: send instructions to the HVAC system.

[0149] Branch 2: Adjust the lights, where branch 2 includes sub-branch 2.1: Query the current light status; sub-branch 2.2: Set the light brightness (e.g., medium brightness); sub-branch 2.3: Send instructions to the lighting system.

[0150] Branch 3: Check the video device, where branch 3 includes sub-branch 3.1: Query device status; sub-branch 3.2: Test video and audio functions; sub-branch 3.3: Send instructions to preheat the device.

[0151] Then, the feasibility and effectiveness of each branch are evaluated, the optimal path is selected, and specific operation tasks are generated based on the selected path.

[0152] In building control, generating operational tasks using thought chaining, based on semantic understanding of natural language statements, is an effective approach. Thought chaining simulates the human thought process, breaking down complex instructions into a series of executable steps. This helps automated systems more clearly understand instructions, break down tasks, and execute them step by step.

[0153] Figure 5 This is a schematic diagram of generating an operation task in a thought chain manner according to an embodiment of the present invention. Figure 5 In the example, after node 72 representing the task definition, nodes 81 to 82 are executed sequentially, and finally node 82 representing the task completion is reached.

[0154] For example, suppose user 10 issues the following instruction to agent 12: "Prepare the meeting room in advance for tomorrow morning's meeting. Ensure the temperature is suitable, the lights are bright, and the projector is ready." Agent 12 sends this instruction to large language model 11. The semantic understanding result of large language model 11 includes the following: Intent: Prepare the meeting room; Time: Tomorrow morning; Conditions: Suitable temperature, moderate lighting, and ready projector.

[0155] Based on the results of semantic understanding, the large language model 11 constructs a thought chain and breaks down the instructions into executable steps: (1) Determine the meeting room: Confirm which meeting room. (2) Set the time: Determine the specific time point for preparation. (3) Adjust the temperature: Set the appropriate temperature according to the season and user preferences. (4) Adjust the lighting: Set the lighting brightness to ensure it is moderate. (5) Check the projector: Ensure that the projector is ready, including the power supply and connection status. (6) Send a notification: Send a reminder to the participants before the meeting starts.

[0156] Based on the thought chain, large language model 11 generates the following operational tasks: Task 1: Query the specific meeting time for tomorrow morning. Task 2: Automatically adjust the conference room temperature to 22°C one hour before the meeting. Task 3: Automatically adjust the conference room lighting to medium brightness half an hour before the meeting. Task 4: Check the projector's power and connection status half an hour before the meeting. Task 5: Send a meeting reminder to all participants 15 minutes before the meeting.

[0157] In one embodiment, the semantic understanding result includes a location and an output type. The operation task includes at least one of the following: reading device data corresponding to the location and associated with the output type from a database; determining output data corresponding to the output type based on the device data corresponding to the location and associated with the output type; and outputting the output data based on a human-computer interaction interface.

[0158] For example, the user inputs: "What is the average temperature on the 4th floor?" The semantic understanding result includes the location (4th floor) and the output type (average temperature). Therefore, the operation tasks include: (1) reading the device data corresponding to the location (4th floor) and associated with the output type (average temperature) from the database (for example, the temperature of all temperature sensors on the 4th floor); based on the device data corresponding to the location and associated with the output type, determining the output data that matches the output type (that is, calculating the average temperature of the 4th floor based on the temperature of all temperature sensors on the 4th floor); and outputting the output data based on the human-computer interaction interface (that is, outputting the average temperature of the 4th floor).

[0159] In one embodiment, the semantic understanding result includes the location and the target value of the controlled parameter; the operation task includes at least one of the following: reading the current value of the controlled parameter at the location from a database; determining whether to adjust the current value to the target value based on the knowledge in the local database invoked by the large language model; if it is determined that the current value is not adjusted to the target value, issuing an error prompt based on the human-computer interaction interface; if it is determined that the current value is adjusted to the target value, sending the target value to the configuration field of the controlled device at the location in the database for controlling the controlled parameter. After the target value is sent to the configuration field of the controlled device, corresponding control of the controlled device can be performed based on the configuration field.

[0160] For example, user input: "Set the air conditioning temperature in room 407 to 24 degrees." The semantic understanding result includes the location (room 407) and the target value (24 degrees) of the controlled parameter (air conditioning temperature). Therefore, the operation task includes:

[0161] (1) Read the current value of the air conditioning temperature corresponding to the location (room 407) from the database; (2) Based on the knowledge in the local database that calls the large language model, determine whether to adjust the current value to the target value (i.e., whether to adjust the current value to 24 degrees); (3) When it is determined that the current value is not adjusted to the target value, issue an error prompt based on the human-computer interaction interface; (4) When it is determined that the current value is adjusted to the target value, send the target value to the configuration field of the controlled device at the location in the database for controlling the controlled parameter. For example, the knowledge in the local database may include: regulations regarding indoor temperature; control constraints regarding air conditioning equipment, etc. After the target value is sent to the configuration field of the controlled device at the location in the database for controlling the controlled parameter, the configuration field is updated, so that the controlled device can automatically adjust the set temperature to the target value based on the updated configuration field.

[0162] In one embodiment, the semantic understanding result includes a location and predetermined keywords; the operation task includes at least one of the following: reading the current value of a controlled parameter associated with the keyword corresponding to the location from a database; determining an adjustment value for the current value based on the knowledge in the local database and the keywords invoked by the large language model; and sending the adjustment value to a configuration field of the controlled device at the location in the database for controlling the controlled parameter. For example, adjectives based on user experience can be set as keywords, such as "cold," "hot," "dry," "humid," and so on.

[0163] For example, user input: "Conference Room 407 is a bit cold." The semantic understanding result includes the location (Conference Room 407) and the keyword (cold). Therefore, the operation tasks include: reading the current value of the controlled parameter (i.e., air conditioning temperature) that matches the location (Conference Room 407) and is associated with the keyword (cold) from the database; determining the adjustment value of the current value based on the knowledge and keywords in the local database that calls the large language model; and sending the adjustment value to the configuration field of the controlled device at the location in the database for controlling the controlled parameter. For example, the knowledge in the local database may include: regulations regarding indoor temperature; control constraints regarding air conditioning equipment, and so on.

[0164] In one embodiment, the semantic understanding results include the location, type and action of the controlled device; the operation task includes at least one of the following: reading the current state of the controlled device at the location and of the type from the database; determining the adjustment state of the current state based on the knowledge and action in the local database that calls the large language model; and sending the adjustment state to the configuration field of the controlled device at the location and of the type in the database.

[0165] For example, user input: "Turn on the lights in the Innovation Center." The semantic understanding results include the location (Innovation Center), the type of controlled device (light), and the action (turn on). Therefore, the operation tasks include: reading the current state of the controlled device (light) at the location (Innovation Center) that meets the type from the database; determining the adjustment state of the current state based on the knowledge and actions in the local database that call the large language model; and sending the adjustment state to the configuration field of the controlled device at the location in the database that meets the type. For example, the knowledge in the local database may include: regulations regarding indoor lighting; control constraints regarding lighting equipment, and so on.

[0166] In the above description, the large language model 11 performs semantic understanding on natural language sentences and generates operation tasks based on the semantic understanding results. In an optional embodiment, the intelligent agent 12 can receive natural language sentences and send the natural language sentences to the large language model 11, and the large language model 11 performs semantic understanding on the natural language sentences. The large language model 11 sends the semantic understanding results to the intelligent agent 12, and the intelligent agent 12 generates operation tasks based on the semantic understanding results. For example, based on the semantic understanding results, the intelligent agent 12 generates operation tasks in the form of a thinking chain based on a linear thinking process or a thinking tree based on a branching structure. Moreover, the intelligent agent 12 also accesses the database 13 to obtain device data associated with the operation task from the building's device data, and performs the operation task based on the device data associated with the operation task. The specific process of the intelligent agent 12 generating an operation task based on the semantic understanding results can refer to the specific process of the large language model 11 generating an operation task, which will not be repeated here.

[0167] In one embodiment, the database 13 is a Haystack database; the large language model 11 is used to generate an operation task containing labels based on the semantic understanding results; the intelligent agent 12 is used to access the Haystack database based on the labels to obtain equipment data associated with the operation task; wherein the labels include at least one of the following: site label; floor label; area label; equipment label; temperature label; humidity label; carbon dioxide label; occupancy label.

[0168] When using a data model like Haystack to store device data, you can create rich tags for each device to better describe the device's properties, status, and behavior. For example: Site Tag: describes the specific building or site where the device or sensor is located, such as "Central Office Building" or "Warehouse 3". Floor Tag: indicates the floor where the device or sensor is located, such as:

[0169] "Floor 5" or "Level 2." Zone Tag: Describes the area or department within a building where a device or sensor is located, for example, "Conference Area," "Server Room," or "Office Space." Equipment Tag: Used to identify a specific device or system, for example, "HVAC System," "Lighting Panel," or "Security Camera." Temperature Tag: A sensor or control device related to temperature, for example, "Temperature Sensor" or "Heating Unit." Humidity Tag: A sensor or control device related to humidity, for example, "Humidity Sensor" or "Dehumidifier 301." CO2 Tag: A sensor or control device related to carbon dioxide concentration, for example, "CO2 Sensor 303" or "Ventilation System 304." Occupancy Tag: A sensor or control device related to occupancy, for example, "Occupancy Sensor 401" or "Access Control 402."

[0170] In one embodiment, there are multiple tags for the operation task; the intelligent agent is used to access the Haystack database one by one using each tag of the operation task in a predetermined tag order to obtain the intermediate device data obtained by each access; based on the intermediate device data obtained by each access, the device data associated with the operation task is determined.

[0171] In one embodiment, the operation task has multiple tags; the intelligent agent is used to combine the multiple tags of the operation task into a tag chain according to a predetermined tag order; and use the tag chain to access the Haystack database to obtain device data associated with the operation task.

[0172] For example, when the user inputs: "What is the average temperature of the 4th floor?", the large language model 11 determines the operation tasks based on the semantic recognition results, including: (1) reading the device data corresponding to the location (4th floor) and associated with the output type (average temperature) from the database (for example, the temperature of each temperature sensor on the 4th floor); based on the device data corresponding to the location and associated with the output type, determining the output data that matches the output type (that is, calculating the average temperature of the 4th floor based on the temperature of each temperature sensor on the 4th floor); and outputting the output data based on the human-computer interaction interface (that is, outputting the average temperature of the 4th floor).

[0173] For the first operation task, the large language model 11 parses the labels including floor: "floor" and temperature:

[0174] Using the tag "floor," database 13 is queried using the keyword "temp." The resulting floors include: BSCE-C7 Floor 1, BSCE-C7 Floor 2, BSCE-C7 Floor 3, BSCE-C7 Floor 4, BSCE-C7 Floor 5, BSCE-C7 Floor 6, and BSCE-C7 Floor 7. Since the query results include four floors, database 13 can be further queried using the tag "temp" to obtain the temperatures of the temperature sensors on all floors. At this point, large language model 11 no longer needs to access database 13. Based on the results of these two calls, the temperatures of all temperature sensors on Floor 4 of BSCE-C7 can be determined.

[0175] For the second operation task, the large language model 11 calculates the average temperature of all temperature sensors in the four layers of BSCE-C7, that is, the average temperature of the four layers.

[0176] For the third operation task, the large language model 11 sends the average temperature of the 4 layers to the intelligent agent 12, so that the intelligent agent 12 returns it to the user 10.

[0177] In the above example, the tags "floor" and "temp" are used to call the database 13 twice. Alternatively, the tags "floor" and "temp" can be combined to form a tag chain "floor+temp" to call the database 13 once.

[0178] Based on the above description, an embodiment of the present invention further proposes a building control method. Figure 6 is an exemplary flow chart of a building control method according to an embodiment of the present invention. Figure 6The method shown can be executed by a controller. The controller can be implemented as an intelligent agent. Optionally, the processor can also be implemented as any one of: a central processing unit (CPU), a graphics processing unit (GPU), a neural network processing unit (NPU), a deep learning processing unit (DPU), an accelerated processing unit (APU), and a general-purpose computing on graphics processing unit (GPGPU). Figure 6 As shown, the method includes:

[0179] Step 101: Receive a natural language sentence about a building.

[0180] Step 102: Input the natural language sentence into the large language model, wherein the large language model performs semantic understanding on the natural language sentence to generate an operation task based on the semantic understanding result.

[0181] Step 103: Access a database storing equipment data of the building to obtain equipment data associated with the operation task.

[0182] Step 104: Execute the operation task based on the device data associated with the operation task.

[0183] In one embodiment, before inputting a natural language sentence into a large language model, the method includes: determining whether the natural language sentence conforms to a predetermined prompt word template, wherein when the natural language sentence conforms to the prompt word template, sending the natural language sentence to the large language model so that the large language model performs semantic understanding on the natural language sentence; when the natural language sentence does not conform to the prompt word template, filtering the natural language sentence; wherein the prompt word template includes at least one of the following: a location field; a device field; a parameter field; a command field; and a keyword field.

[0184] In one embodiment, the database is a Haystack database; generating an operation task based on the semantic understanding result includes: generating an operation task including a tag based on the semantic understanding result; accessing a database for storing equipment data of a building to obtain equipment data associated with the operation task includes: accessing the Haystack database based on the tag to obtain equipment data associated with the operation task; wherein the tag includes at least one of the following: site tag; floor tag; area tag; equipment tag; temperature tag; humidity tag; carbon dioxide tag; occupancy tag.

[0185] In one embodiment, the number of tags for the operation task is multiple; accessing the Haystack database based on the tags to obtain device data associated with the operation task includes: according to a predetermined tag sequence, using each tag of the operation task to access the Haystack database one by one to obtain intermediate device data obtained by each access; determining the device data associated with the operation task based on the intermediate device data obtained by each access; or, according to a predetermined tag sequence, combining multiple tags of the operation task into a tag chain; using the tag chain to access the Haystack database to obtain device data associated with the operation task.

[0186] The embodiment of the present invention further provides a building control device. Figure 7 FIG is an exemplary structural diagram of a building control device according to an embodiment of the present invention. Figure 7 As shown, the building control device 200 includes: a receiving module 201, which is used to receive natural language sentences about buildings; an input module 202, which is used to input the natural language sentences into a large language model, wherein the large language model performs semantic understanding on the natural language sentences to generate operation tasks based on the semantic understanding results; an access module 203, which is used to access a database that stores equipment data of the building to obtain equipment data associated with the operation task; and an execution module 204, which is used to execute the operation task based on the equipment data associated with the operation task.

[0187] An embodiment of the present invention further provides an electronic device having a processor-memory architecture. Figure 8 : is an exemplary structural diagram of an electronic device according to an embodiment of the present invention. Figure 8As shown, the electronic device 300 includes a processor 301, a memory 302, and a computer program stored in the memory 302 and executable on the processor 301. When the computer program is executed by the processor 301, any of the above building control methods is implemented. Specifically, the memory 302 can be implemented as a variety of storage media, such as an electrically erasable programmable read-only memory (EEPROM), a flash memory (Flash memory), or a programmable read-only memory (PROM). The processor 301 can be implemented as one or more central processing units (CPUs) or one or more field programmable gate arrays (FPGAs), wherein the FPGAs integrate one or more CPU cores. Specifically, the CPU or CPU core can be implemented as a CPU, an MCU, a DSP, or the like.

[0188] It should be noted that not all steps and modules in the above processes and structure diagrams are required, and certain steps or modules can be omitted based on actual needs. The execution order of the steps is not fixed and can be adjusted as needed. The division of the modules is merely for the convenience of describing the functional division adopted. In actual implementation, a module can be implemented by multiple modules, and the functions of multiple modules can be implemented by the same module. These modules can be located in the same device or in different devices.

[0189] The hardware modules in each embodiment can be implemented mechanically or electronically. For example, a hardware module may include a specially designed permanent circuit or logic device (such as a dedicated processor, such as an FPGA or ASIC) for performing a specific operation. The hardware module may also include a programmable logic device or circuit (such as a general-purpose processor or other programmable processor) temporarily configured by software to perform a specific operation. As for whether to implement the hardware module mechanically, or using a dedicated permanent circuit, or using a temporarily configured circuit (such as configured by software), it can be decided based on cost and time considerations.

[0190] The above are only preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A building automation system, characterized in that: include: A database (13) for storing equipment data of the building; An intelligent agent (12) for receiving a natural language statement about the building; A large language model (11) is used to perform semantic understanding on the natural language sentence and generate an operation task based on the semantic understanding result; The intelligent agent (12) is further configured to access the database (13) to obtain device data associated with the operation task from the device data of the building, and execute the operation task based on the device data associated with the operation task.

2. The building automation system according to claim 1, wherein: The intelligent agent (12) includes a pre-set prompt word template; The intelligent agent (12) is configured to, when the natural language sentence conforms to the prompt word template, send the natural language sentence to the large language model (11) so that the large language model (11) performs semantic understanding on the natural language sentence; and, when the natural language sentence does not conform to the prompt word template, filter the natural language sentence; The prompt word template includes at least one of the following: a location field; a device field; a parameter field; Command field; keyword field.

3. The building automation system according to claim 1, wherein: The semantic understanding result includes position and output type; The operation task includes at least one of the following: Reading device data corresponding to the location and associated with the output type from the database (13); determining output data that matches the output type based on the device data associated with the output type that matches the location; The output data is output based on a human-computer interaction interface.

4. The building automation system according to claim 1, wherein: The semantic understanding results include the position and target values of the controlled parameters; The operation task includes at least one of the following: reading the current value of the controlled parameter at the location from the database (13); Determining whether to adjust the current value to the target value based on the knowledge in the local database of the large language model (11); When it is determined that the current value is not adjusted to the target value, an error prompt is issued based on the human-computer interaction interface; When it is determined that the current value is adjusted to the target value, the target value is sent to a configuration field of a controlled device at the location in the database (13) for controlling the controlled parameter.

5. The building automation system according to claim 1, wherein: The semantic understanding result includes a location and predetermined keywords; The operation task includes at least one of the following: Reading from the database (13) the current value of the controlled parameter associated with the keyword corresponding to the location; Determining an adjustment value of the current value based on the knowledge in the local database of the large language model (11) and the keyword; The adjustment value is sent to a configuration field of a controlled device at the location in the database (13) for controlling the controlled parameter.

6. The building automation system according to claim 1, wherein: The semantic understanding result includes the location, type and action of the controlled device; The operation task includes at least one of the following: Reading the current status of the controlled device at the location that meets the type from the database (13); Determining an adjustment state of the current state based on the knowledge in the local database of the large language model (11) and the action; The adjustment status is sent to the configuration field of the controlled device of the type at the location in the database (13).

7. The building automation system according to any one of claims 1 to 6, characterized in that: The database (13) is a Haystack database; The large language model (11) is used to generate an operation task containing a label based on the semantic understanding result; The intelligent agent (12) is configured to access the Haystack database based on the tag to obtain device data associated with the operation task; wherein the tag comprises at least one of the following: Site tag; floor tag; area tag; equipment tag; temperature tag; humidity tag; carbon dioxide tag; occupancy tag.

8. The building automation system according to claim 7, characterized in that: The number of tags of the operation task is multiple; The intelligent agent (12) is used to access the Haystack database one by one using each tag of the operation task in accordance with a predetermined tag sequence to obtain the intermediate device data obtained by each access; Determining the device data associated with the operation task based on the intermediate device data obtained during each access; or The intelligent agent (12) is used to combine multiple tags of the operation task into a tag chain according to a predetermined tag sequence; and use the tag chain to access the Haystack database to obtain device data associated with the operation task.

9. The building automation system according to any one of claims 1 to 6, characterized in that: The large language model (11) is used to generate the operation task based on the semantic understanding result in a thinking chain based on a linear thinking process or a thinking tree based on a branch structure.

10. A building control method, characterized in that: include: receiving (101) a natural language sentence about a building; Inputting (102) the natural language sentence into a large language model, wherein the large language model performs semantic understanding on the natural language sentence to generate an operation task based on the semantic understanding result; Accessing (103) a database storing equipment data of the building to obtain equipment data associated with the operation task; The operational task is performed (104) based on the device data associated with the operational task.

11. The building control method according to claim 10, characterized in that: Prior to inputting (102) the natural language sentence into the large language model, the method includes: determining whether the natural language sentence conforms to a predetermined prompt word template, wherein when the natural language sentence conforms to the prompt word template, sending the natural language sentence to the large language model so that the large language model performs semantic understanding on the natural language sentence; and when the natural language sentence does not conform to the prompt word template, filtering the natural language sentence; The prompt word template includes at least one of the following: a location field; a device field; a parameter field; a command field; and a keyword field.

12. The building control method according to claim 10 or 11, characterized in that: The database is a Haystack database; The generating of the operation task based on the semantic understanding result includes: generating the operation task including the label based on the semantic understanding result; The accessing (103) a database for storing equipment data of a building to obtain equipment data associated with the operation task includes: accessing the Haystack database based on the tag to obtain equipment data associated with the operation task; wherein the tag comprises at least one of the following: Site tag; floor tag; area tag; equipment tag; temperature tag; humidity tag; carbon dioxide tag; occupancy tag.

13. The building control method according to claim 12, characterized in that: The number of tags of the operation task is multiple; Accessing the Haystack database based on the tag to obtain device data associated with the operation task includes: Accessing the Haystack database one by one using each tag of the operation task in a predetermined tag sequence to obtain the intermediate device data obtained in each access; determining the device data associated with the operation task based on the intermediate device data obtained during each access; or According to a predetermined tag sequence, multiple tags of the operation task are combined into a tag chain; and the tag chain is used to access the Haystack database to obtain device data associated with the operation task.

14. A building control device, characterized in that: include: A receiving module (201) is used to receive a natural language sentence about a building; An input module (202) is used to input the natural language sentence into a large language model, wherein the large language model performs semantic understanding on the natural language sentence to generate an operation task based on the semantic understanding result; An access module (203) is used to access a database storing equipment data of the building to obtain equipment data associated with the operation task; An execution module (204) is configured to execute the operation task based on the device data associated with the operation task.

15. An electronic device, characterized in that: comprising a processor (301) and a memory (302); The memory (302) stores an application program that can be executed by the processor (301), and is used to enable the processor (301) to execute the building control method according to any one of claims 10 to 13.

16. A computer-readable storage medium having computer instructions stored thereon, characterized in that: When the computer instructions are executed by a processor, the building control method according to any one of claims 10 to 13 is implemented.

17. A computer program product, characterized in that The invention comprises a computer program, which implements the building control method according to any one of claims 10 to 13 when executed by a processor.

Images