Method and apparatus for querying building data and synthetic data

Abstract

Embodiments of the present disclosure disclose a method and device for querying building data and synthetic data. The specific implementation of the method comprises: extracting a graph embedding vector from input building-related graph data, and storing the graph embedding vector into a vector database and storing the graph data into a graph database; converting a question related to the graph data into a question embedding vector, and searching for a target vector similar to the question embedding vector in the vector database; generating a query instruction in a predetermined language based on the target vector through a first large language model; executing the query instruction in the graph database to determine a query result; and outputting a natural language answer to the question based on the query result through a second large language model. The implementation realizes efficient querying of complex building data and reduces the user's dependence on professional knowledge of semantic technology.

Description

Technical Field

[0001] The embodiments disclosed herein relate to the field of computer technology, specifically to methods and apparatus for querying building data and synthetic data, applicable to the architecture, engineering, construction, and operations (AECO) industry. Background Technology

[0002] The AECO industry generates massive amounts of complex, heterogeneous data, such as Building Information Models (BIM), sensor data, and equipment information. This data is typically represented using semantic technologies (such as Brick Schema and ifcOWL) to enable the integration and management of multi-source data. However, due to the complexity and diversity of the data, and the need for specialized languages ​​like SPARQL to construct effective queries, querying this type of data remains a significant challenge. Traditionally, only personnel with expertise in semantic technologies can efficiently extract relevant information from these data sets.

[0003] The data sources in the AECO industry are diverse, including Building Information Modeling (BIM), sensor data, equipment specifications, and maintenance records. These data sources vary, with different formats, standards, and naming conventions, making data integration challenging. Therefore, even basic data queries often require significant time and effort. Inconsistencies between different data sources hinder efficient data utilization, leading to delayed and inefficient decision-making processes, ultimately reducing the overall operational efficiency of the industry.

[0004] To address these challenges, the AECO industry has been committed to adopting Semantic Web technology in recent years, which provides a standardized framework for data integration and querying, demonstrating significant potential in overcoming data fragmentation. Recently, ASHRAE's 223P ontology has also been released to optimize the representation of semantic data in building systems. However, despite the promising prospects of these semantic frameworks, non-technical professionals (such as architects, engineers, project managers, and owners) still face significant barriers—they struggle to understand and write the SPARQL (SPARQL Protocol and RDF Query Language) language required to query these ontologies. This complexity hinders the widespread adoption of semantic technologies and limits their practical utility within the AECO industry. Summary of the Invention

[0005] Embodiments of this disclosure provide methods and apparatus for querying building data and synthetic data.

[0006] In a first aspect, embodiments of this disclosure provide a method for querying building data, comprising: extracting graph embedding vectors from input building-related graph data and storing the graph embedding vectors in a vector database, storing the graph data in a graph database; converting questions related to the graph data into question embedding vectors and searching for target vectors similar to the question embedding vectors in the vector database; generating a query instruction in a predetermined language based on the target vectors using a first major language model; executing the query instruction in the graph database to determine the query results; and outputting a natural language answer to the question based on the query results using a second major language model.

[0007] Secondly, embodiments of this disclosure provide a method for synthesizing data, comprising: reasoning on a building information model to determine a reasoning model supplemented with superclasses and inverse relationships; filtering the reasoning model according to predetermined conditions to select target data; generating a query instruction in a predetermined language based on the target data using a predefined template; generating an original question in natural language based on the query instruction using a third language model; executing the query instruction in a graph database to determine a baseline truth value; and generating a dataset based on the original question and the baseline truth value, wherein the dataset is used to evaluate the accuracy of any of the methods in the first aspect.

[0008] Thirdly, embodiments of this disclosure provide a device for querying building data, comprising: a first embedding unit configured to extract graph embedding vectors from input building-related graph data, store the graph embedding vectors in a vector database, and store the graph data in a graph database; a second embedding unit configured to convert questions related to the graph data into question embedding vectors, and search for target vectors similar to the question embedding vectors in the vector database; an instruction generation unit configured to generate query instructions in a predetermined language based on the target vectors and using a first large language model; an instruction execution unit configured to execute the query instructions in the graph database and determine the query results; and a result output unit configured to output the natural language answer to the question based on the query results and using a second large language model.

[0009] Fourthly, embodiments of this disclosure provide an apparatus for synthesizing data, comprising: an inference unit configured to infer from a building information model to determine an inference model supplemented with superclasses and inverse relationships; a filtering unit configured to filter the inference model according to predetermined conditions to select target data; an instruction generation unit configured to generate query instructions in a predetermined language based on the target data and using a predefined template; a question generation unit configured to generate original questions in natural language based on the query instructions and using a third language model; a query unit configured to execute the query instructions in a graph database to determine baseline truth values; and a sample generation unit configured to generate a dataset based on the original questions and baseline truth values, wherein the dataset is used to evaluate the accuracy of any of the methods in the first aspect.

[0010] Fifthly, embodiments of this disclosure provide an electronic device, including: one or more processors; and a storage device having one or more computer programs stored thereon, which, when executed by the one or more processors, cause the one or more processors to perform the method as described in any one of the first or second aspects.

[0011] In a sixth aspect, embodiments of this disclosure provide a computer-readable medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the method as described in any one of the first or second aspects.

[0012] In a seventh aspect, embodiments of this disclosure provide a computer program product including a computer program that, when executed by a processor, implements the method as described in any one of the first or second aspects.

[0013] This disclosure provides a novel natural language query method for building data based on large language models (LLMs) and retrieval-augmented generation (RAG) technologies, specifically designed for the architecture, engineering, construction, and operations (AECO) industry. This method aims to solve the query challenges of complex semantic data models, particularly those based on building-specific ontologies such as Brick Schema. The core innovation is a hybrid retrieval mechanism called Vector-Graph Retrieval-Augmented Generation (VG-RAG), which combines vector embeddings with graph database queries to provide context-relevant information for generating accurate SPARQL queries. VG-RAG overcomes the limitations of traditional RAG methods (such as tokens and context constraints) and improves LLM performance without relying on large domain-specific training datasets. Furthermore, this application proposes a synthetic data generation framework capable of creating a controlled evaluation environment for benchmarking RAG technologies in queries of varying complexity and edge scenarios. The system achieves query accuracy improvements of up to 40% compared to traditional methods, providing a reliable solution for non-technical users in the AECO industry. This technology enables efficient querying of complex building data, reducing users' reliance on SPARQL or semantic technology expertise.

[0014] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0015] Other features, objects, and advantages of this disclosure will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1 This is an exemplary system architecture diagram to which one embodiment of this disclosure can be applied; Figure 2 This is a flowchart of one embodiment of the method for querying building data according to this disclosure; Figure 3 This is a schematic diagram of the HNSW hierarchical structure and similarity search path according to this disclosure; Figure 4 This is a flowchart of one embodiment of the method for synthesizing data according to this disclosure; Figure 5 This is a schematic diagram illustrating an application scenario of the method for synthesizing data according to this disclosure; Figure 6 This is a schematic diagram illustrating the performance of the method for evaluating and querying building data according to this disclosure; Figure 7 This is a schematic diagram of an embodiment of a device for querying building data according to the present disclosure; Figure 8 This is a schematic diagram of an embodiment of the apparatus for synthesizing data according to the present disclosure; Figure 9 This is a schematic diagram of a computer system for implementing an electronic device according to an embodiment of the present disclosure. Detailed Implementation

[0016] The present disclosure will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings.

[0017] It should be noted that, unless otherwise specified, the embodiments and features described in this disclosure can be combined with each other. This disclosure will now be described in detail with reference to the accompanying drawings and embodiments.

[0018] Figure 1 An exemplary system architecture is shown that can be applied to embodiments of the building data querying method or apparatus of this disclosure.

[0019] like Figure 1 As shown, the system architecture may include terminal devices and servers. Users can use terminal devices to upload architecture-related graph data (e.g., architectural semantic web data) and ask questions, receiving answers in natural language. The process consists of four stages: the first stage is graph data initialization, extracting relevant information from the input data and generating embeddings, which are then stored in a vector database and a graph database; the second stage is similarity search, using the embedded question vectors to retrieve the top K most relevant entities, relations, and ontology from the vector database; the third stage is suggestion enhancement, using the retrieved data to optimize suggestions, enabling the large language model to generate SPARQL query commands that meet the requirements of graph data queries; the final stage is natural language response generation, combining the SPARQL query results with the original question to generate a coherent natural language answer through the language model.

[0020] It should be noted that the method for querying building data provided in the embodiments of this disclosure is generally executed by a server, and correspondingly, the device for querying building data is generally located in the server.

[0021] It should be understood that Figure 1 The number of terminal devices and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices and servers can be used.

[0022] Continue to refer to Figure 2The diagram illustrates a flow 200 of an embodiment of a method for querying building data according to the present disclosure. This method for querying building data includes the following steps: Step 201: Extract graph embedding vectors from the input building-related graph data, store the graph embedding vectors in the vector database, and store the graph data in the graph database; In this embodiment, entity text, relation text, and ontology text can be extracted from graph data, and then converted into embedding vectors respectively to generate graph embedding vectors. Both the vector database and the graph database can be existing open-source databases.

[0023] Step 202: Convert the questions related to the graph data into question embedding vectors, and search the vector database for target vectors similar to the question embedding vectors; In this embodiment, the similarity search method may employ locality-sensitive hashing, inverted file index (IVF), and hierarchical navigable small-world (HNSW) graphs. The embedding vectors of graphs with high similarity are named the target vectors.

[0024] Step 203: Based on the target vector, generate a query command in the predetermined language using the first major language model; In this embodiment, the first large language model is a pre-trained neural network, which can be an existing large language model such as ChatGPT, LLaMA, deepseek, and Qianwen. Based on a predetermined prompt template, query instructions in a predetermined language (e.g., SPARQL) are generated based on information such as entities, relations, and ontology in the target vector.

[0025] Step 204: Execute the query command in the graph database to determine the query results; In this embodiment, the query results, i.e. the answer to the question, are searched in the graph database according to the query command, although they may include technical terms such as prefixes.

[0026] Step 205: Based on the query results, output the natural language answer to the question using the second language model.

[0027] In this embodiment, the query results are described in technical language and need to be converted into natural language answers using a second large language model. The second large language model is a pre-trained neural network, which can be an existing large language model such as ChatGPT, LLaMA, deepseek, or Qianwen. The second large language model can be the same as or different from the first large language model.

[0028] In some optional implementations of this embodiment, the graph embedding vector is extracted from the input building-related graph data, including: traversing the triple data in the graph data to extract the ontology set, entity set, and relation set; and embedding the ontology set, entity set, and relation set respectively to obtain the graph embedding vector.

[0029] After receiving the input graph data, it is loaded into the vector database and the graph database for initialization. For the vector database, the triple data (entity, relation, entity) is first traversed to obtain the entity set. and relation set Furthermore, the ontology set is extracted from the (entity, rdf:type, ontology) triple. After obtaining the entity set E, relation set R, and ontology set O, they are embedded and stored in a vector database. During embedding, the open-source Sentence-BERT model is used: "multi-qa-mpnet-base-dot-v1" is used for entity and ontology embedding, and "all-mpnet-base-v2" is used for relation embedding. In recent performance evaluations, "all-mpnet-base-v2" ranks first in sentence embedding tasks, while "multi-qa-mpnet-base-dot-v1" performs best in semantic search tasks. After embedding, the data is stored in the Chroma vector database, which uses a hierarchical navigable small-world (HNSW) structure to ensure efficient storage and retrieval (see Table 1 for an example of a vector database).

[0030]

[0031] Table 1. Examples of embedding vector data for graph components.

[0032] For graph database storage, the Python library "rdflib" is used to load the graph data into memory so that information can be retrieved using SPARQL query commands.

[0033] In some optional implementations of this embodiment, searching for target vectors similar to the question embedding vector in the vector database includes: constructing a multi-level graph structure based on the vector database, wherein the bottom layer of the graph structure contains all data points, and the number of data points decreases layer by layer from the bottom layer to the top layer of the graph structure; and searching for a predetermined number of target vectors most similar to the question embedding vector in the graph structure based on the hierarchical navigable small world (HNSW) algorithm.

[0034] Similarity search in vector databases is crucial in various applications such as information retrieval, recommender systems, and machine learning. Commonly used similarity search methods include Locality Sensitive Hashing (LSH), Inverted File Index (IVF), and Hierarchical Navigable Small World (HNSW) graphs. This application chooses the HNSW algorithm for similarity search because it strikes a good balance between search accuracy and computational efficiency in high-dimensional spaces. Furthermore, mainstream vector databases such as Chroma, Milvus, and Weaviate also employ HNSW for vector similarity search.

[0035] The HNSW algorithm requires constructing a graph structure before the search. This graph uses a hierarchical structure, with the bottom layer (layer 0) containing all data points, and the number of data points decreasing with each subsequent layer. In each layer, nodes represent data points, and edges represent neighbor relationships based on vector similarity, such as... Figure 3 As shown.

[0036] Figure 3 This not only showcases the graph structure of HNSW but also demonstrates the search process. After inputting the question (represented by yellow dots), the similarity search begins at the starting point (blue dot) of the entry layer. The algorithm finds the closest data point to the query in the current layer and then continues searching to the next layer. This process is repeated until the top K most similar data points to the query are found at layer 0, completing the similarity search. The red nodes and edges clearly annotate this search path. Through this process, for a given question Q, the K most relevant entities, relations, and ontology to Q can be obtained: : , and .

[0037] In some optional implementations of this embodiment, generating a query instruction in a predetermined language based on the target vector and through a first large language model includes: extracting a prefix from the graph data; embedding the prefix and the ontology, entities, and relations involved in the target vector into a predefined first prompt template to determine first prompt information, wherein the first prompt template includes a task description for indicating the specific requirements for generating the query instruction in the predetermined language and an example of the query instruction in the predetermined language; and generating the query instruction in the predetermined language based on the first prompt information and through the first large language model.

[0038] The generated SPARQL query hints can be enhanced using a first hint template. The hint enhancement process aims to improve the accuracy of generated SPARQL statements through structured templates. This process uses a predefined hint template as a framework, incorporating core components related to question Q (i.e.,...) , , and The most relevant entities, relationships, and ontology represented are systematically integrated into the template. The template contains a clear and concise task description, explicitly defining the specific requirements for SPARQL query generation. Furthermore, the template provides multiple examples to support few-shot learning, enabling the model to learn from existing queries and adjust its response.

[0039] Prefixes are unique identifiers used to locate the source of data in graph databases, so query commands need to be generated based on the prefixes.

[0040] The first hint template is shown below. Integrating these components into the hints guides the model to construct SPARQL statements that accurately reflect the provided entity and ontology relationships and structure. Ultimately, this enhancement process ensures that the generated SPARQL queries are both accurate and problem-specific, thereby improving data retrieval efficiency and interaction with architectural semantic web resources. After this process is complete, a SPARQL query that meets the specific requirements of question Q will be obtained.

[0041]

[0042] In some optional implementations of this embodiment, based on the query results, the natural language answer to the question is output through a second large language model, including: embedding the query results and the question into a predefined second prompt template, determining second prompt information, wherein the second prompt template includes a task description for indicating that prefixes in the query results should be ignored; and based on the second prompt information, the natural language answer to the question is output through a second large language model.

[0043] After executing a SPARQL query on a graph database, the results are processed to generate a natural language response. Specifically, the SPARQL query results are embedded in a predefined hint template to guide the Large Language Model (LLM) in generating a clear and understandable answer. The second hint template is shown below, where the placeholder {question} is replaced with the user's original question Q, and {context} is replaced with the retrieval data obtained from the SPARQL execution. The hint instructs the LLM to ignore all prefixes in the SPARQL results, ensuring that the response is generated only based on relevant information. In this way, the final answer is presented in a clear and understandable form, improving the user's comprehension of the query results.

[0044]

[0045] Synthetic data has become an important solution for addressing data scarcity, privacy concerns, and the need for large-scale, controlled experiments. One of the core advantages of synthetic data is the flexibility in scene generation; researchers can design datasets with specific characteristics or simulate various real-world scenarios by adjusting parameters. This application uses synthetic data as the primary evaluation basis because it can create databases of varying complexity based on predefined scenarios. Specifically, we constructed six different databases, each corresponding to a different level of complexity—gradually transitioning from simple queries to more challenging ones. This approach systematically validates the performance of the proposed method at different difficulty levels and supports comprehensive quantitative analysis. By using synthetic data in this way, we can directly evaluate the accuracy of the method under various controlled conditions, ensuring the reliability and reproducibility of the results.

[0046] The input to this application is a natural language problem, and evaluating the accuracy of the output requires consideration of multiple factors. One possible approach is to evaluate accuracy by generating SPARQL queries, but since different SPARQL queries may produce the same results, direct comparison of query statements is not feasible. Another approach is to evaluate the generated natural language response, but the variability of language expression also presents challenges. Therefore, we ultimately chose to perform exact matching on the results obtained by executing SPARQL queries on a graph database. Accordingly, the dataset consists of natural language problems and their corresponding SPARQL query result pairs.

[0047] Further reference Figure 4 The diagram illustrates a flow 400 of an embodiment of a method for synthesizing data. Flow 400 of the method for synthesizing data includes the following steps: Step 401: Reasoning is performed on the building information model to determine the reasoning model supplemented with superclasses and inverse relationships; Step 402: Filter the inference model according to predetermined conditions to select target data; Step 403: Based on the target data, generate query instructions in a predefined language using a predefined template; Step 404: Based on the query command, generate the original question in natural language using the third major language model; Step 405: Execute a query command in the graph database to determine the baseline truth value; Step 406: Generate a dataset based on the original problem and the benchmark truth.

[0048] Figure 5This paper demonstrates the synthetic data generation process employed in this application. The process first infers from the Brick Information Model (Brick Model for short), then filters the data under specific conditions using six different filters (i.e., selection criteria). Subsequently, SPARQL queries are generated using predefined templates. These SPARQL queries serve a dual purpose: first, to add annotations to natural language questions through a large language model; and second, to execute queries on a graph database to obtain SPARQL query results. Finally, natural language question-SPARQL query result pairs are generated as the basis for evaluation.

[0049] In step 401, reasoning is performed on the Building Information Model (BIM). Reasoning on the Brick model significantly improves its semantic depth by automatically deriving and supplementing information, reducing the need for manual input. During reasoning, the model handles superclass and inverse relationships. Examples of these two types of reasoning are shown below. In superclass reasoning, when a Brick entity is instantiated from a specific class, the reasoning mechanism automatically associates it with the relevant superclass. For example, if an entity is classified as "brick:Air_Temperature_Sensor," the reasoning mechanism recognizes that it is also an instance of the parent classes "brick:Temperature_Sensor," "brick:Sensor," and "brick:Point." In inverse relationship reasoning, the reasoning mechanism establishes mutually inverse relationships in the model. For example, if an entity is associated with another entity through a "brick:isPointOf" relationship, the reasoning mechanism creates a reciprocal "brick:hasPoint" relationship.

[0050]

[0051] After reasoning, the implicit data was made explicit, greatly enriching the data volume of the Brick model and laying a solid foundation for the subsequent generation of synthetic data.

[0052] Steps 402-403 involve data filtering and SPARQL query generation.

[0053] The main task is to filter data that meets specific conditions from the inferred Brick model and generate corresponding SPARQL queries based on predefined templates. Through this process, SPARQL statements are extracted from the actual Brick model.

[0054] The inference model is filtered according to predetermined conditions to select target data. Six datasets can be generated in the following ways: 1. Filtering ontology belonging to a specific category from the inference model; 2. Counting the number of ontology belonging to a specific category from the inference model; 3. Filtering the type or category associated with a specific entity from the inference model; 4. Filtering other entities directly related to a specific entity from the inference model; 5. Filtering other entities indirectly related to a specific entity through a two-step relationship from the inference model; 6. Filtering entities that satisfy a specific relationship with a specific entity and belong to a specific ontology category from the inference model.

[0055] Table 2 shows the dataset description, filtering conditions, and corresponding SPARQL query templates.

[0056]

[0057] Table 2 The symbols in the filter (i.e., the filtering conditions) are defined as follows: onto represents the ontology, enti represents the entity, r represents the relation, s represents the subject, o represents the object, G represents the graph data, and T represents the triple.

[0058] Table 2 details the information for each dataset. The “Dataset” column lists the dataset names, from dataset 1 to dataset 6, with increasing data depth (i.e., an increasing number of entities) and a corresponding increase in complexity. The “Category” column includes two types: ontology-based and entity-based. In ontology-based queries, the final SPARQL statement only contains the ontology and related relations, without involving entity information; in entity-based queries, the SPARQL statement includes entities, directly querying for specific entities. The “Description” column describes the problem for each dataset. “Complexity” is quantified based on the number and type of attributes included in the SPARQL query for each dataset. We assigned a specific complexity value to each attribute to represent its relative difficulty to Large Language Models (LLMs). Attributes provided by the Brick Schema (such as ontology onto and relation r) are assigned a complexity value of 1 because the schema is publicly accessible and may already be included in the LLM training data. Conversely, entities enti are scenario-specific and need to be extracted from project-specific Brick models (and are generally not included in the LLM training data), thus being assigned a complexity value of 2. The total complexity of each dataset is determined by summing the complexity values ​​of all attributes in its filter criteria. For example, the filter criteria for dataset 4 involve entities enti and relations r, resulting in a total complexity of 3 (enti contributes 2, r contributes 1). This quantification method reflects the increasing difficulty of SPARQL queries as the number of attributes increases. The "Filter Criteria" column specifies the criteria for extracting data that meet specific standards, using set notation. For example, the filter criteria for dataset 4... This represents the set of A's that satisfy condition B. The symbol (enti, r) represents data stored as entity-relation pairs, where enti, r, and o are elements in the Brick graph G. (enti, r, o) ∈ T, r ≠ rdf:type indicates that entity-relation pairs are extracted from triples, excluding rdf:type relations. The "Template" column contains the data extracted from the filter criteria as input. For example, in dataset 4... <prefix>The prefixes required for SPARQL statements, This indicates projecting entity attributes from set F. This indicates the projection relationship attribute.

[0059] Ultimately, these generated SPARQL statements are used to create Natural Language Question-SPARQL Query Result Pairs to meet evaluation requirements.

[0060] Steps 404-405 generate Natural Language Question-SPARQL query result pairs.

[0061] To generate a dataset for evaluating our method, we take natural language questions as input and SPARQL query results as output. Therefore, for each SPARQL statement generated in the previous section, we perform two simultaneous steps: first, we generate a natural language question using a predefined template and ChatGPT-4o; second, we execute a SPARQL query on a graph database to obtain the corresponding query results. By synchronizing these two steps, we ensure that each natural language question directly corresponds to its SPARQL query result.

[0062] Natural language problems are generated by transforming SPARQL queries into descriptive representations. To automate and scale this process, we designed custom templates for each database, containing natural language instructions and example questions for few-shot learning. Hints generated from these templates are input into ChatGPT-4o for batch annotation of SPARQL queries, as shown below.

[0063]

[0064] This method generates a dataset of natural language question-SPARQL query result pairs as input-output labels for testing and evaluating the method. This dataset effectively correlates natural language queries with structured query results, enabling a comprehensive evaluation of the method's ability to interpret natural language and retrieve precise information from graph databases.

[0065] This application utilizes Python to implement the algorithms and conduct experiments throughout the entire workflow. Specific tasks include Brick model preprocessing, synthetic data generation, VG-RAG algorithm development, RAG comparative analysis, and large language model deployment. The research dataset contains 45 Brick models from the Mortar dataset. Semantic reasoning for knowledge graph processing employs the "owlrl" method from the "brickschema" package. The "rdflib" package is used to manage the Brick models as a graph database, enabling data storage and retrieval. Vector data storage and retrieval utilize the "langchain_chroma" package to construct a vector database. Furthermore, the "langchain_huggingface" package is used to deploy and invoke embedded models to assist in vector generation. The "ollama" package is used to deploy the Llama 3.1 model, and the "langchain_openai" package is used to invoke OpenAI's GPT-4o model. The VG-RAG architecture is implemented based on the "langchain" package. To optimize computational efficiency, the "concurrent.futures" package is used for parallel processing during experiments. Finally, "streamlit" is used to develop the web application. All experiments and calculations were performed on a Linux server equipped with an Intel Xeon Platinum 8260 CPU (24 cores and 48 threads, base frequency 2.4 GHz, turbo frequency 3.9 GHz) and eight Nvidia RTX 4090 GPUs (each GPU is equipped with 24GB of GDDR6X video memory, for a total of 192 GB of video memory).

[0066] This application generates six datasets containing 18,555 examples by integrating data from 45 real-world Brick models. These datasets are intended to serve as benchmark sets for evaluating Natural Language Query Semantic Web data. The complexity of the datasets increases progressively from low to high, as detailed in Table 3.

[0067]

[0068] Table 3

[0069] The datasets are sized as follows: Dataset 1 and Dataset 2 each contain 256 examples; Dataset 3 contains 5020 examples; Dataset 4 contains 5136 examples; Dataset 5 contains 4427 examples; and Dataset 6 contains 3460 examples. This difference in size stems from the varying numbers of entities, relations, and ontology elements in the model. Specifically, Dataset 1 focuses on retrieving all instances of a specific class; Dataset 2 counts the number of instances of a specific class; Dataset 3 retrieves the type or class associated with a specific entity; Dataset 4 obtains related entities through specific relations; Dataset 5 retrieves indirectly related entities through two-step relations; and Dataset 6 retrieves related entities through specific relations and ensures they belong to a specific ontology class.

[0070] We used these six datasets to evaluate the accuracy of the VG-RAG method in BuildingGPT. Each dataset served as a benchmark to evaluate BuildingGPT's ability to generate correct SPARQL queries based on specific instructions and inputs, as shown in Table 4:

[0071] Table 4

[0072] Table 4 presents a sample from the dataset, containing several key fields. "Instructions" describes the task of guiding the model to generate SPARQL queries in the Brick Schema, providing context for the query generation process and serving as the basis for model fine-tuning. "Inputs" represents the user's query input, specifying the concrete request the model needs to process. "Outputs" are the correct SPARQL queries, used as a comparison reference (although not directly used for accuracy verification, as multiple SPARQL queries may return the same result, it still provides a correct representation of the task). "Brick Model" refers to the model (Turtle format) used as input to the RAG method, defining the entities and relationships interacted with by the SPARQL query, enabling the system to retrieve correct data from the graph database. "Results" are the output returned by the graph database after executing the SPARQL query, crucial for accuracy verification—if the SPARQL query result generated by BuildingGPT matches the "Results" field in the dataset, the query is considered correct. These datasets will be used for subsequent evaluation of the accuracy of various models and the RAG method.

[0073] In some optional implementations of this embodiment, the method further includes: extracting graph embedding vectors from the building information model and storing the graph embedding vectors in a vector database, and storing the building information model in a graph database; setting the hit count to 0, traversing the original questions in the dataset, and repeatedly performing the following steps: converting the original questions into question embedding vectors, and searching for target vectors similar to the question embedding vectors in the vector database; generating a query instruction in a predetermined language based on the target vectors through a first large language model; executing the query instruction in the graph database to determine the query result; incrementing the hit count by 1 in response to the query result completely matching the baseline truth value corresponding to the original question; and determining the accuracy of the method for querying building data based on the ratio of the hit count to the number of original questions in the dataset.

[0074] This application evaluates the accuracy of Large Language Models (LLMs) generating SPARQL queries under different Retrieval Enhancement Generation (RAG) frameworks. Specifically, the effectiveness of the proposed VG-RAG method is verified by comparison with two control groups. The purpose of setting up control groups is to explore whether incorporating external knowledge retrieval can improve the accuracy of SPARQL query generation. The three experimental configurations for comparison are: (1) LLM without any retrieval enhancement (control group 1); (2) LLM based on LangChain enhancement (control group 2); (3) LLM based on VG-RAG enhancement (experimental group).

[0075] like Figure 6 As shown, the core difference between the control group and the experimental group lies in the retrieval mechanism used to improve the query generation capability of the language model.

[0076] In control group 1, the LLM generates SPARQL queries solely based on natural language input (Q). This group does not utilize any external knowledge retrieval; the query generation process relies entirely on the model's pre-trained knowledge. The setup without retrieval augmentation provides a benchmark for evaluating the role of additional external information.

[0077] Control group 2 introduced the LangChain framework, which is designed for developing applications that leverage language models to perform tasks such as question answering, document retrieval, and summarization, and natively supports SPARQL-based graph data retrieval. Specifically, LangChain improves the accuracy of query generation by incorporating all ontology terms extracted from the massive knowledge base of the Brick model into the suggestions.

[0078] In contrast, the experimental group used the proposed VG-RAG method. VG-RAG retrieval is more targeted, extracting only the top K most relevant entities directly related to the input query (Q). ), ontology ( ) and relationships ( Furthermore, VG-RAG retrieves SPARQL syntax-related prefixes to ensure that the generated queries are not only context-accurate but also syntactically correct. This targeted retrieval enables VG-RAG to generate more precise and domain-specific SPARQL queries, potentially achieving higher accuracy than other groups.

[0079] To evaluate the accuracy of the generated SPARQL queries, we compared the query results with predefined baseline truth values. The baseline truth values ​​for the example queries are shown in the "Results" field of Table 4. The primary metric for accuracy evaluation is Exact Match Accuracy, commonly referred to as Hits@1. This metric measures the proportion of optimal results generated by the LLM that perfectly match the correct answers in the baseline truth values. Hits@1 is widely used in information retrieval to evaluate the effectiveness of retrieval and generation processes. The formula for calculating Exact Match Accuracy (Hits@1) is as follows:

[0080] Where N is the total number of queries in the test set; 1( ) is an indicator function that returns 1 if the result generated by the LLM matches the baseline truth value of a query, and 0 otherwise. The SPARQL query result generated by LLM for query i; The correct result for querying i is obtained from the baseline truth data.

[0081] The Hits@1 metric quantifies the ability of three different models to generate accurate SPARQL queries. A higher Hits@1 value indicates that the query generated by the model is more likely to return the correct result on the first attempt, thus reflecting the effectiveness of the retrieval augmentation generation method in improving query accuracy.

[0082] This application provides a detailed comparison of the performance of three LLMs (ChatGPT-4o, LLaMA 3.1-70B, and LLaMA 3.1-8B) under three different RAG frameworks. The analysis aims to verify the effectiveness of the proposed VG-RAG framework in improving the accuracy of SPARQL query generation and compares it with two benchmark methods (LLM without retrieval enhancement and LLM based on LangChain retrieval enhancement).

[0083] ChatGPT-4o is a large language model developed by OpenAI, based on the Transformer architecture, and performs exceptionally well in handling complex reasoning tasks, understanding long texts, and analyzing contextual relationships. LLaMA 3.1 is a language model developed by Meta, with parameter sizes ranging from 8 billion to 70 billion. LLaMA 3.1-70B is a large model with high accuracy in handling complex language tasks; while LLaMA 3.1-8B is a smaller version in the same series, with limited ability to handle large-scale complex tasks.

[0084] Table 5 summarizes the Hits@1 exact match accuracy and average performance of each model across the six datasets. Datasets 1 and 2 represent simple queries, datasets 3 and 4 represent medium-complexity queries, and datasets 5 and 6 represent more complex queries.

[0085]

[0086] Table 5

[0087] 1. ChatGPT-4o Performance

[0088] Without retrieval augmentation, ChatGPT-4o achieved high accuracy on simple datasets (Dataset 1 and Dataset 2), at 98.83% and 96.09%, respectively. However, as the complexity of Datasets 3 through 6 increased, the model's performance significantly decreased, with accuracy fluctuating between 43.01% and 8.74%. This decline indicates that the model cannot acquire domain-specific knowledge beyond the pre-training data. With LangChain, ChatGPT-4o achieved near-perfect accuracy on simple datasets (Dataset 1 and Dataset 2), but its performance plateaued on medium and complex datasets. The addition of LangChain only slightly improved accuracy; the model still struggled with complex queries, with accuracy on Datasets 3 through 6 ranging from only 7.5% to 42%. Introducing VG-RAG significantly improved ChatGPT-4o's performance across all datasets. It achieved near-perfect accuracy, particularly on simple datasets, and performed exceptionally well on medium-complexity queries (Dataset 3 and Dataset 4), achieving accuracies of 91.81% and 89.43%, respectively. Even on more complex queries (datasets 5 and 6), the model outperformed the baseline model and the LangChain-enhanced model, achieving accuracies of 54.13% and 30.43%, respectively. The mean accuracy of 77.57% represents a significant improvement over the no-RAG and LangChain RAG configurations.

[0089] 2. Performance of LLaMA 3.1-70B

[0090] LLaMA 3.1-70B achieved moderate accuracy on simple datasets (92.58% on dataset 1), but its performance dropped rapidly on more complex queries (only 3.46% on dataset 6). This indicates that while the model has great potential, the lack of retrieval augmentations makes it difficult to effectively handle complex queries. Combining with LangChain improved the performance of LLaMA 3.1-70B on simple datasets, but it still performed poorly on complex queries. The model performed stably on datasets 1 and 2, but struggled to break the 40% accuracy barrier on complex queries, reflecting the challenges of using general retrieval to handle domain-specific SPARQL generation. Adding VG-RAG significantly improved the accuracy of LLaMA 3.1 (70B) on all datasets, especially on medium and complex queries. The model performed exceptionally well on datasets 3 and 4, with accuracies of 94.62% and 92.50%, respectively, and also achieved significant improvements on more difficult datasets, with 56.76% accuracy on dataset 5 and 20.78% on dataset 6. The average performance of 74.32% demonstrates the effectiveness of VG-RAG in improving the accuracy of generating complex SPARQL queries.

[0091] 3. Performance of LLaMA 3.1-8B

[0092] LLaMA 3.1-8B has limited ability to generate accurate SPARQL queries, with accuracy approaching 0% on most datasets. This reflects the challenge of small models handling complex query tasks without retrieval augmentations. Combining with LangChain RAG slightly improves the performance of LLaMA 3.1-8B, but it still struggles with complex queries, with low accuracy on datasets 3 through 6. The average accuracy of 18.40% indicates that LangChain's extensive retrieval mechanism is insufficient to compensate for the limitations of model size. The VG-RAG framework improves the accuracy of LLaMA 3.1-8B, especially on simple and moderately complex datasets (Dataset 1: 97.27%, Dataset 3: 94.92%, Dataset 4: 88.43%). However, model performance drops significantly with increasing query complexity. Specifically, the accuracy is 46.56% on Dataset 5, 16.08% on Dataset 6, and the average accuracy is 57.21%. It is worth noting that regardless of the RAG method used, LLaMA 3.1-8B achieved an accuracy of 0% on Dataset 2. This may be because Dataset 2 is a counting problem, which has a different query type than other datasets. Furthermore, the pre-training data for LLaMA 3.1 (8B) may lack sufficient data relevant to this problem type.

[0093] The results show that the VG-RAG framework achieves significant performance improvements across all tested Large Language Models (LLMs). By incorporating retrieval enhancements, VG-RAG enables models to generate more accurate and domain-specific SPARQL queries, especially on medium to complex datasets. Notably, ChatGPT-4o, incorporating VG-RAG, exhibits the highest accuracy at 77.57%, a 21.99% improvement over the non-RAG benchmark. LLaMA 3.1-70B also benefits significantly from VG-RAG, showing substantial performance improvements in medium and complex queries, achieving an overall accuracy of 74.32%, a 24.57% improvement over the non-RAG method. Although LLaMA 3.1-8B has the lowest accuracy (57.21%), it still represents a 39.36% improvement over the non-RAG method. However, its overall performance is limited by the relatively small model size, particularly on dataset 2.

[0094] To further validate the effectiveness of the proposed VG-RAG framework, we conducted a case study using a real-world dataset from a building management system. Specifically, we focused on the "bldg4.ttl" data file, which represents a building containing a heating, ventilation, and air conditioning (HVAC) system, encompassing multiple components such as HVAC zones, air handling units (AHUs), and variable air volume (VAVs). In this case study, five different natural language queries were input into the model based on LangChain RAG and VG-RAG enhancements. The results were compared to evaluate the ability of the two methods to generate accurate and context-relevant data. These queries were designed to test the model's ability to retrieve and interpret specific data points related to the HVAC system. Table 6 summarizes the responses to human input, those generated by LangChain RAG and VG-RAG, and the correctness of the responses based on benchmark truth judgments.

[0095]

[0096] Table 6

[0097] Table 6 compares the results of LangChain RAG and VG-RAG on five questions. For the first query, both frameworks successfully listed the HVAC zones mentioned in the project, providing the same accurate answer. However, for subsequent questions, the performance of the two methods differed significantly. LangChain RAG failed to answer four of the five questions, returning either incomplete responses or no relevant information. For example, when asked "Which devices supply air to RM1109?", LangChain RAG returned an empty response, indicating a lack of information about the relationship between RM1109 and other entities in the system. In contrast, VG-RAG provided the correct answer, identifying "VAVRM1109" as the air supplier for RM1109 based on the dataset. Similarly, LangChain RAG failed to provide valid information about the functionality of VAVRM1109, while VG-RAG correctly identified it as a key component of the HVAC system—the Variable Air Volume (VAV) housing. The query "What measurement points are there for VAVRM1109?" further illustrates this difference. LangChain RAG returned a response indicating missing data, while VG-RAG accurately listed the relevant measurement points, including air temperature, flow rate, and other system variables. Finally, when asked, "What is the time series ID of the air temperature setpoint in zone bldg4 AHU01 RM1109?", LangChain RAG again failed to provide the correct ID, while VG-RAG successfully retrieved the specific time series identifier, demonstrating its superior ability to handle complex queries and retrieve relevant time-series data.

[0098] We also documented the process of generating answers through direct interaction with the front end, as shown below, demonstrating two key parts of the experiment: (a) the results of the LangChain RAG framework; and (b) the results of the VG-RAG framework.

[0099]

[0100] (a) Shows the output of the LangChain RAG framework. This framework first returns a "query" (the original question posed by the user) and a "context." "result" refers to the answer generated by the system based on its internal knowledge, and "sparql_query" is the SPARQL query generated by the model to retrieve relevant data. (b) Shows the output of the VG-RAG framework. This framework provides a "response" (the actual answer generated based on the relevant context) and a "sparql_query" (the SPARQL query generated by the system). Furthermore, "sparql_result" represents the output obtained by executing this SPARQL query on the database.

[0101] Experimental results clearly demonstrate that the performance of SPARQL query generation in LLM is directly affected by the retrieval mechanism employed. Specifically, comparing the baseline LLM (without retrieval augmentation) with the LangChain RAG augmented model reveals almost no improvement in accuracy. This is because LangChain's approach incorporates the entire ontology from the graph database into the prompt. Since ontology types such as BrickSchema and ifcOWL are typically already included in the pre-training data of LLM, they fail to provide new and useful information. Furthermore, incorporating the entire ontology can lead to excessively long prompts, exceeding the model's maximum token limit. Consequently, important information may be truncated, negatively impacting the accuracy of the generated SPARQL queries.

[0102] In contrast, the proposed VG-RAG framework significantly improves accuracy by providing the model with only concise and relevant information. This approach is not limited by any specific LLM; regardless of the model used, VG-RAG consistently and significantly improves the accuracy of SPARQL query generation. Across different models, VG-RAG outperforms the baseline model and the LangChain RAG augmented model by 20% to 40%. This highlights the effectiveness of the VG-RAG hybrid retrieval mechanism, which combines graph and vector databases to extract key, context-specific information. By providing the model with concise, domain-specific knowledge, VG-RAG significantly improves the accuracy of SPARQL query generation.

[0103] An interesting finding from the experiments is that different LLMs exhibit performance variations under different RAG configurations. Regardless of the configuration, ChatGPT-4o consistently outperforms LLaMA 3.1-70B and LLaMA 3.1-8B, consistent with previous research demonstrating the ChatGPT series models' superior generalization ability in complex natural language understanding and generation tasks.

[0104] Notably, the LLaMA 3.1-70B model demonstrated excellent performance in the Retrieval Augmentation (RAG) task. While its accuracy was slightly lower than ChatGPT-4o, the difference was only about 3%. ChatGPT-3 has 175 billion parameters, and although the exact number of parameters for ChatGPT-4o has not been publicly disclosed, it is clearly much larger than ChatGPT-3. In contrast, LLaMA 3.1-70B has only 70 billion parameters, significantly fewer than the former two. This indicates that even with a smaller model size, LLaMA 3.1-70B can achieve excellent results with VG-RAG augmentation. Furthermore, LLaMA 3.1-70B can be deployed locally, providing greater flexibility for practical applications.

[0105] A key innovation of this application is the use of synthetic data to evaluate the LLM, which includes "instruction" and "output" fields and can also serve as a training set for fine-tuning. By generating synthetic SPARQL queries with accurate answers, we are able to evaluate the model in a controlled and repeatable environment. This approach not only reduces reliance on manually labeled data but also allows us to systematically study the impact of different configurations on model accuracy. Synthetic datasets have proven to be a valuable resource for benchmarking models across a wide range of query types and complexities. Importantly, the synthetic nature of the data also allows us to simulate edge cases. For example, in Dataset 2, we generated counting problems, which differ from those in other datasets, and the results show that the LLaMA 3.1-8B model struggles to solve such problems.

[0106] Furthermore, our synthetic data generation process is based on 45 real Brick models, providing a large number of real queries not seen in the LLM training data. Experiments show that we generate many queries that are not seen in most large language models (LLMs), and the VG-RAG-enhanced model can significantly improve the accuracy of these unseen queries.

[0107] This application proposes an innovative framework that utilizes Large Language Models (LLMs) and Retrieval Augmentation Generation (RAG) techniques to achieve natural language queries on semantic web building data. The proposed Vector-Graph Retrieval Augmentation Generation (VG-RAG) pipeline addresses key challenges in semantic querying, particularly applicable to ontologies related to the building industry. The main contributions of this application are summarized below: 1. Natural Language Query Method for Semantic Building Data: This paper proposes an efficient method for querying complex building information models using natural language. The VG-RAG framework can directly generate accurate SPARQL queries from natural language, improving the accessibility and usability of semantic web building data. This method is particularly valuable to professionals in the AECO industry, allowing them to query data flexibly without requiring specialized technical knowledge.

[0108] 2. Development of the VG-RAG pipeline: The VG-RAG framework combines vector-based and graph-based retrieval, significantly improving query generation accuracy by 20% to 40% compared to benchmark methods. This framework demonstrates superior performance across various LLMs (ChatGPT-4o, LLaMA 3.1-70B, and LLaMA 3.1-8B), outperforming traditional RAG frameworks such as LangChain.

[0109] 3. Synthetic Data for Evaluation: A novel synthetic data framework was developed for evaluating the RAG method at different complexity levels. The model performance was comprehensively evaluated by generating six datasets of varying complexities (containing 18,555 samples).

[0110] 4. Improved Accuracy: The VG-RAG framework achieved an accuracy of 77.57%. The enhanced model achieved an accuracy of 90% on four datasets, demonstrating the effectiveness of the hybrid retrieval mechanism in improving the accuracy of SPARQL query generation.

[0111] In summary, the VG-RAG framework provides a reliable solution for natural language querying in the AECO industry. By improving the accuracy of SPARQL queries and providing domain-specific knowledge, this framework significantly advances the interoperability and accessibility of AECO domain semantic data.

[0112] Further reference Figure 7 As an implementation of the methods shown in the above figures, this disclosure provides an embodiment of a device for querying building data, which is similar to... Figure 2 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0113] like Figure 7 As shown, the device 700 for querying building data in this embodiment includes: a first embedding unit 701, a second embedding unit 702, an instruction generation unit 703, an instruction execution unit 704, and a result output unit 705. The first embedding unit 701 is configured to extract graph embedding vectors from input building-related graph data and store the graph embedding vectors in a vector database, and store the graph data in a graph database. The second embedding unit 702 is configured to convert questions related to the graph data into question embedding vectors and search for target vectors similar to the question embedding vectors in the vector database. The instruction generation unit 703 is configured to generate a query instruction in a predetermined language based on the target vectors using a first large language model. The instruction execution unit 704 is configured to execute the query instruction in the graph database to determine the query result. The result output unit 705 is configured to output a natural language answer to the question based on the query result using a second large language model.

[0114] In this embodiment, the specific processing of the first embedding unit 701, the second embedding unit 702, the instruction generation unit 703, the instruction execution unit 704, and the result output unit 705 of the device 700 for querying building data can be referred to Figure 2 The corresponding steps are 201, 202, 203, 204 and 205 in the embodiment.

[0115] In some optional implementations of this embodiment, the first embedding unit 701 is further configured to: traverse the triplet data in the graph data, extract the ontology set, entity set and relation set; and perform embedding processing on the ontology set, entity set and relation set respectively to obtain the graph embedding vector.

[0116] In some optional implementations of this embodiment, the second embedding unit 702 is further configured to: construct a multi-level graph structure based on a vector database, wherein the bottom layer of the graph structure contains all data points, and the number of data points decreases layer by layer from the bottom layer to the top layer of the graph structure; and search for a predetermined number of target vectors in the graph structure that are most similar to the question embedding vector based on the hierarchical navigable small world (HNSW) algorithm.

[0117] In some optional implementations of this embodiment, the instruction generation unit 703 is further configured to: extract a prefix from the graph data; embed the prefix and the ontology, entity and relation involved in the target vector into a predefined first prompt template to determine first prompt information, wherein the first prompt template includes a task description for indicating the specific requirements for generating a query instruction in a predetermined language and an example of a query instruction in the predetermined language; and generate a query instruction in the predetermined language based on the first prompt information and a first large language model.

[0118] In some optional implementations of this embodiment, the result output unit 705 is further configured to: embed the query results and the question into a predefined second prompt template, determine the second prompt information, wherein the second prompt template includes a task description for indicating that prefixes in the query results should be ignored; and output the natural language answer to the question based on the second prompt information and through a second large language model.

[0119] In some optional implementations of this embodiment, the first and second largest language models adopt ChatGPT-4o.

[0120] Further reference Figure 8 As an implementation of the methods shown in the above figures, this disclosure provides an embodiment of a device for synthesizing data, which is similar to... Figure 4 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0121] like Figure 8 As shown, the data synthesis apparatus 800 of this embodiment includes: a reasoning unit 801, a filtering unit 802, an instruction generation unit 803, a question generation unit 804, a query unit 805, and a sample generation unit 806. Specifically, the reasoning unit 801 is configured to perform reasoning on the Building Information Model (BIM) to determine a reasoning model supplemented with superclasses and inverse relationships; the filtering unit 802 is configured to filter the reasoning model according to predetermined conditions to select target data; the instruction generation unit 803 is configured to generate query instructions in a predetermined language based on the target data and using a predefined template; the question generation unit 804 is configured to generate original questions in natural language based on the query instructions and using a third language model; the query unit 805 is configured to execute the query instructions in a graph database to determine a baseline truth value; and the sample generation unit 806 is configured to generate a dataset based on the original questions and the baseline truth value.

[0122] In this embodiment, the specific processing of the inference unit 801, filtering unit 802, instruction generation unit 803, question generation unit 804, query unit 805, and sample generation unit 806 of the data synthesis device 800 can be referred to Figure 4 Steps 401-406 in the corresponding embodiment.

[0123] In some optional implementations of this embodiment, the filtering unit 802 is further configured to perform at least one of the following operations: filtering ontologies belonging to a specific category from the inference model; counting the number of ontologies belonging to a specific category from the inference model; filtering types or categories associated with a specific entity from the inference model; filtering other entities associated with a specific entity through a direct relationship from the inference model; filtering other entities indirectly associated with a specific entity through a two-step relationship from the inference model; and filtering entities that satisfy a specific relationship with a specific entity and belong to a category of a specific ontology from the inference model.

[0124] In some optional implementations of this embodiment, the apparatus 800 further includes an evaluation unit (not shown in the figures), configured to: extract graph embedding vectors from the building information model and store the graph embedding vectors in a vector database, and store the building information model in a graph database; set the hit count to 0, traverse the original questions in the dataset, and repeatedly perform the following steps: convert the original questions into question embedding vectors and search for target vectors similar to the question embedding vectors in the vector database; based on the target vectors, generate a query instruction in a predetermined language through a first large language model; execute the query instruction in the graph database to determine the query result; and increment the hit count by 1 in response to the query result completely matching the baseline truth value corresponding to the original question. The accuracy of the method for querying building data is determined by the ratio of the number of hits to the number of original questions in the dataset.

[0125] It should be noted that the collection, gathering, updating, analysis, processing, use, transmission, and storage of user personal information involved in this disclosed technical solution all comply with relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. Necessary measures are taken to prevent unauthorized access to user personal information data and to safeguard user personal information security, network security, and national security.

[0126] According to embodiments of this disclosure, this disclosure also provides an electronic device and a readable storage medium.

[0127] An electronic device includes: one or more processors; and a storage device having one or more computer programs stored thereon, which, when executed by the one or more processors, cause the one or more processors to implement the method described in process 200 or 400.

[0128] A computer-readable medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the method described in process 200 or 400.

[0129] Figure 9 A schematic block diagram of an example electronic device 900 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0130] like Figure 9 As shown, device 900 includes a computing unit 901, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 902 or a computer program loaded from storage unit 908 into random access memory (RAM) 903. RAM 903 may also store various programs and data required for the operation of device 900. The computing unit 901, ROM 902, and RAM 903 are interconnected via bus 904. Input / output (I / O) interface 905 is also connected to bus 904.

[0131] Multiple components in device 900 are connected to I / O interface 905, including: input unit 906, such as keyboard, mouse, etc.; output unit 907, such as various types of monitors, speakers, etc.; storage unit 908, such as disk, optical disk, etc.; and communication unit 909, such as network card, modem, wireless transceiver, etc. Communication unit 909 allows device 900 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0132] The computing unit 901 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 901 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 901 performs the various methods and processes described above, such as road planning methods. For example, in some embodiments, the road planning method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 908. In some embodiments, part or all of the computer program may be loaded and / or installed on device 900 via ROM 902 and / or communication unit 909. When the computer program is loaded into RAM 903 and executed by the computing unit 901, one or more steps of the road planning method described above may be performed. Alternatively, in other embodiments, the computing unit 901 may be configured to perform the road planning method by any other suitable means (e.g., by means of firmware).

[0133] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0134] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0135] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0136] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0137] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0138] Computer systems can include clients and servers. Clients and servers are generally geographically separated and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be servers in distributed systems or servers incorporating blockchain technology. Servers can also be cloud servers, or intelligent cloud computing servers or intelligent cloud hosts with artificial intelligence technology.

[0139] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0140] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.< / prefix>

Claims

1. A method for querying building data, comprising: Graph embedding vectors are extracted from the input building-related graph data, and the graph embedding vectors are stored in a vector database, while the graph data is stored in a graph database. The questions related to the graph data are converted into question embedding vectors, and target vectors similar to the question embedding vectors are searched in the vector database. Based on the target vector, a query command in a predetermined language is generated using the first major language model; The query instruction is executed in the graph database to determine the query results; Based on the query results, the natural language answer to the question is output using the second language model.

2. The method according to claim 1, wherein, The step of extracting graph embedding vectors from the input building-related graph data includes: Traverse the triple data in the graph data to extract the ontology set, entity set, and relation set; The ontology set, the entity set, and the relation set are respectively embedded to obtain a graph embedding vector.

3. The method according to claim 1, wherein, The step of searching the vector database for target vectors similar to the question embedding vector includes: A multi-level graph structure is constructed based on the vector database, wherein the bottom layer of the graph structure contains all data points, and the number of data points decreases layer by layer from the bottom layer to the top layer of the graph structure. Based on the hierarchical navigable small-world (HNSW) algorithm, a predetermined number of target vectors that are most similar to the problem embedding vector are searched in the graph structure.

4. The method according to claim 1, wherein, The step of generating a query command in a predetermined language based on the target vector and using the first large language model includes: Extract the prefix from the graph data; The prefix and the ontology, entity and relation involved in the target vector are embedded into a predefined first prompt template to determine the first prompt information, wherein the first prompt template includes a task description for indicating the specific requirements for generating query instructions in a predetermined language and an example of query instructions in the predetermined language; Based on the first prompt information, a query command in the predetermined language is generated using the first major language model.

5. The method according to claim 4, wherein, Based on the query results, the second major language model is used to output the natural language answer to the question, including: The query results and the question are embedded into a predefined second prompt template to determine the second prompt information, wherein the second prompt template includes a task description for indicating that prefixes in the query results should be ignored; Based on the second prompt information, the natural language answer to the question is output through the second major language model.

6. The method according to claim 1, wherein, The first and second largest language models adopted ChatGPT-4o.

7. A method for synthesizing data, comprising: Reasoning was performed on the building information model to determine a reasoning model supplemented with superclasses and inverse relationships; The inference model is filtered according to predetermined conditions to select target data; Based on the target data, query instructions in a predetermined language are generated using a predefined template; Based on the query command, the original question in natural language is generated using the third major language model; Execute the query command in the graph database to determine the baseline truth value; A dataset is generated based on the original problem and the benchmark truth values, wherein the dataset is used to evaluate the accuracy of the method according to any one of claims 1-6.

8. The method according to claim 7, wherein, The step of filtering the inference model according to predetermined conditions to select target data includes at least one of the following: Filter out ontologies belonging to a specific category from the inference model; The number of ontologies belonging to a specific category is counted from the inference model; Filter out the types or categories associated with a specific entity from the reasoning model; Filter out other entities that are directly related to a specific entity from the reasoning model; Filter out other entities that are indirectly associated with a specific entity through a two-step relationship from the reasoning model; Entities that satisfy a specific relationship with a specific entity and belong to a specific ontology category are selected from the reasoning model.

9. The method according to claim 7, wherein, The method further includes: Graph embedding vectors are extracted from the building information model and stored in a vector database, while the building information model is stored in a graph database. Set the hit count to 0, iterate through the original questions in the dataset, and repeat the following steps: convert the original questions into question embedding vectors, and search for target vectors similar to the question embedding vectors in the vector database; based on the target vectors, generate a query instruction in a predetermined language using the first large language model; execute the query instruction in the graph database to determine the query result; in response to the query result completely matching the baseline truth value corresponding to the original question, increment the hit count by 1. The accuracy of the method for querying building data is determined based on the ratio of the number of hits to the number of original questions in the dataset.

10. A device for querying building data, comprising: The first embedding unit is configured to extract a graph embedding vector from the input building-related graph data, store the graph embedding vector in a vector database, and store the graph data in a graph database; The second embedding unit is configured to convert questions related to the graph data into question embedding vectors and search the vector database for target vectors similar to the question embedding vectors. The instruction generation unit is configured to generate a query instruction in a predetermined language based on the target vector and through a first large language model; The instruction execution unit is configured to execute the query instruction in the graph database and determine the query result; The result output unit is configured to output the natural language answer to the question based on the query result, using a second language model.

11. An apparatus for synthesizing data, comprising: The reasoning unit is configured to reason about the building information model and determine a reasoning model supplemented with superclasses and inverse relationships; The filtering unit is configured to filter the inference model according to predetermined conditions to filter out target data; The instruction generation unit is configured to generate query instructions in a predetermined language based on the target data and using a predefined template. The question generation unit is configured to generate a natural language question based on the query instruction using a third language model. The query unit is configured to execute the query instruction in the graph database to determine the baseline truth value; A sample generation unit is configured to generate a dataset based on the original problem and the benchmark truth value, wherein the dataset is used to evaluate the accuracy of the method according to any one of claims 1-6.

12. An electronic device, comprising: One or more processors; Storage device, on which one or more computer programs are stored, When the one or more computer programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-9.

13. A computer-readable medium having a computer program stored thereon, wherein, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-9.

Images