Multi-modal knowledge graph enhanced base model-based inhomogeneous indoor environment optimization design decision-making method for tall and large space building

Abstract

The invention provides a high and large space building non-uniform indoor environment optimization design decision-making method based on a multi-mode knowledge graph enhanced base model. In the design decision making process, a green performance optimization design model conforming to the typical engineering scene of the non-uniform indoor environment of the tall and large space building is constructed, the problem of non-uniform indoor environment optimization design is understood, and a method for optimization design decision making is provided. And outputting building morphological parameters, space parameters, heating ventilation and air conditioning system parameters and other numerical values, and green performance image data of the typical section of the tall and large space building and the like. On this basis, the invention provides a construction method of a knowledge representation enabled enhanced base model, which realizes optimal allocation of computing power resources based on an end-cloud collaborative distributed working architecture, improves knowledge retrieval and question and answer efficiency in cooperation with a multi-modal knowledge graph and a vector database, reduces calculation cost and time, and improves calculation precision. And in combination with a digital sand table, visual interaction of an optimization design scheme is realized.

Description

Technical Field

[0001] The present invention belongs to the technical field of green building performance optimization design, and in particular relates to a non-uniform indoor environment optimization design decision method for tall and large space buildings based on a multimodal knowledge graph enhanced base model. Background Art

[0002] Faced with higher requirements for building energy conservation, traditional optimization design decision-making methods rely on performance simulation of green performance data, which has certain limitations in computational efficiency and computational cost. It is difficult to process multi-modal building green performance data at the same time, and cannot effectively form multi-platform collaboration of design and simulation. The energy-saving benefits cannot meet new development needs and cannot effectively respond to the multi-dimensional requirements of buildings.

[0003] The indoor environment of tall and large-space buildings is non-uniform. Affected by the coupling of multiple factors, the air flow process is complex, and the environmental parameters are variable, which makes it easy to have problems such as heat accumulation and poor air flow. For different functional areas in this type of building, there are large differences in ventilation, heat load, and personnel density. Some functional areas are prone to local stagnation of personnel, affecting the comfort of users. How to achieve effective temperature and humidity, ventilation control and user comfort adjustment in tall and large-space buildings is the core issue of green performance optimization decision-making for this type of building. Traditional design decision-making methods are difficult to adapt to local conditions and cannot effectively respond to higher comfort and building energy-saving requirements.

[0004] The optimization design decisions for the non-uniform indoor environment of tall and large-space buildings are currently mainly based on physical models and machine learning models, which have the following defects:

[0005] 1. It is difficult to process and integrate multi-source, multi-modal heterogeneous data in non-uniform indoor environments. Traditional methods rely on static data and models, cannot provide feedback on real-time data, lack the ability to respond to emergencies, slow design response, delayed decision-making, and lack flexibility and adaptability;

[0006] 2. It is difficult to accurately simulate and analyze the green performance data affected by multiple coupling factors in the non-uniform indoor environment of large-scale buildings. Traditional methods lack knowledge reasoning capabilities, cannot effectively leverage the advantages of cloud computing power, have poor learning capabilities for historical cases and similar solutions, and are difficult to effectively propose personalized design decisions for specific scenarios. They also have poor flexibility and scalability.

[0007] In recent years, multimodal large language models (MM-LLMs) have shown great potential in multimodal data fusion processing. Based on inputs of numerical, text, image, video, audio and other modalities, large models can understand user needs and generate decision-making recommendations and technical reports, but the hallucination problem of large models restricts the accuracy and reliability of their application in engineering scenario design. The use of knowledge graph enhancement can effectively improve this problem. Relying on knowledge graphs that fit specific engineering scenarios, the generation model is enhanced by retrieval, matching design problems with triples in the knowledge graph, and enabling the intelligent question-answering process of large models. Summary of the invention

[0008] The purpose of the present invention is to solve the problems in the prior art and proposes a method for optimizing the design of non-uniform indoor environments of tall and large space buildings based on a multimodal knowledge graph enhanced base model.

[0009] The present invention is implemented by the following technical scheme. The present invention proposes a non-uniform indoor environment optimization design decision method for tall and large space buildings based on a multimodal knowledge graph enhanced base model. The method comprises the following steps:

[0010] S1, obtain multimodal data knowledge of non-uniform indoor environment;

[0011] The step S1 comprises the following steps:

[0012] S11. Collect green performance data of non-uniform indoor environment;

[0013] S12, generating an embedded representation of the green performance information of the fused multi-modal non-uniform indoor environment;

[0014] S13, extracting green performance data information of non-uniform indoor environment;

[0015] S2, integrate the multimodal data knowledge of non-uniform indoor environment and construct the multimodal knowledge graph and vector database of non-uniform indoor environment;

[0016] The step S2 comprises the following steps:

[0017] S21, converting the extracted information into RDF triples;

[0018] S22, constructing a multimodal knowledge graph for non-uniform indoor environments based on RDF triples;

[0019] S23. Determine the elements of typical non-uniform indoor environment engineering scenes;

[0020] S24. Build and update the corresponding multimodal knowledge graph and vector database;

[0021] S3. Strengthen the knowledge retrieval of optimization design decision-making problems and build a green performance optimization design model that fits typical engineering scenarios of non-uniform indoor environments;

[0022] The step S3 comprises the following steps:

[0023] S31. Modularize the knowledge in the knowledge graph and obtain a structured subgraph;

[0024] S32, understand design problems and improve knowledge retrieval efficiency based on structured subgraphs;

[0025] S33. Construct a green performance optimization design model that fits typical engineering scenarios of non-uniform indoor environments;

[0026] S4, end-cloud collaboration, flexibly handles optimization design problems, and realizes visual interaction of optimization design results in the form of a digital sandbox;

[0027] The step S4 comprises the following steps:

[0028] S41. Build a terminal-cloud collaborative architecture that meets typical engineering scenarios of various non-uniform indoor environments;

[0029] S42. Build a digital twin model of a tall and large-space building, and display the optimized design results through a digital sandbox to achieve visual interaction.

[0030] Furthermore, in the step S13, information extraction includes entity extraction, relationship extraction and event extraction; the entity extraction refers to the application of named entity recognition method based on BiLSTM-CRF deep learning model, using context information to extract specific building green performance objects or concepts from unstructured non-uniform indoor environment green performance data set; the relationship extraction refers to the use of OpenIE and BERT relational classification model or dependency syntax analysis method to identify the relationship between the above-mentioned non-uniform indoor environment green performance entities; the event extraction refers to the spatiotemporal dimension parameters of multiple non-uniform indoor environment green performance entities, which are detected by ACE event extraction framework, and the non-uniform indoor environment green performance event trigger words and related information are identified based on the deep learning model.

[0031] Furthermore, in the step S21, the non-uniform indoor environment green performance data is converted into RDF triples of "entity-relationship-entity" through Python's RDF toolkit RDFLib, and the triples are stored in a graph database for subsequent processing and query operations;

[0032] In the step S22, the extracted entities are aligned with the existing entities of the non-uniform indoor environment, a relationship graph between the entities is constructed, and the entities are linked using a graph matching algorithm; for the records pointing to the unified entity in the non-uniform indoor environment knowledge base, the distance calculation of the entities and relationships embedded in the vector space is merged through a clustering algorithm or a DeepWalk algorithm; the relationship similarity is calculated using an embedding model, and the relationship between the same entities from different data sources is integrated using a graph neural network; the integrated data is reasoned based on rules, according to priority rules set based on the reliability of the data source and the timestamp, to resolve conflicting or contradictory knowledge obtained from different data sources and derive new knowledge; based on machine learning reasoning, the implicit green performance data relationship of the non-uniform indoor environment is mined, knowledge enhancement is performed, and the accuracy of information extraction and the reasoning ability of the knowledge graph are improved.

[0033] Furthermore, in step S24, the extracted non-uniform indoor environment multimodal data is preprocessed; the temperature, humidity, air velocity, light intensity and energy consumption real-time monitoring numerical structured data are normalized and dimensionally reduced; the project information, user needs, design specifications and standards, and policy document text data are based on the natural language processing model to generate embedded vectors; the building photos, renderings, general plans, floor plans, sections, elevations, detail drawings and optimized simulation results pseudo-color image data are extracted using ResNet or CNN models, and the user behavior video and building three-dimensional model video data are extracted from key frames using C3D networks and LSTM models;

[0034] In step S24, based on the generated embedding vector, the feature dimension of green performance data is standardized; an HNSW vector index structure is constructed to improve the speed of vector similarity search, and support is provided for the matching of semi-structured data in the multimodal knowledge graph, thereby accelerating the intelligent question-answering process for user design questions;

[0035] In step S24, for the green performance evaluation indicators of typical engineering scenarios, further determine the influencing factors, including equipment distribution, wall insulation performance, natural lighting, personnel flow and dynamic load; and clarify the typical models or algorithm elements in the calculation or simulation, including CFD model, sunshine analysis model, dynamic energy consumption analysis model, PMV model, PPD model and energy consumption optimization algorithm, regional zoning control algorithm;

[0036] In step S24, a green performance portrait of the non-uniform indoor environment of a specific large-space engineering scene is constructed, and a multimodal green performance database of the specific large-space engineering scene is completed and updated.

[0037] Furthermore, in step S31, a community detection algorithm is used to identify mutually related node groups and segment the knowledge graph into different communities; a graph embedding algorithm is used to identify and add missing relationships or entities to enhance the integrity of the knowledge graph and assist knowledge reasoning.

[0038] Furthermore, in step S33, based on the multimodal knowledge graph that fits the typical engineering scenario of green performance optimization design of non-uniform indoor environment, on the cloud platform, based on the base model, an enhanced base model enabled by the green performance knowledge representation of non-uniform indoor environment is trained to give full play to the advantages of cloud computing power, process the coupling relationship between multimodal data involved in the task of optimizing the design of non-uniform indoor environment, and enable the decision-making model based on the subjective experience of the designer.

[0039] Further, in step S41, based on the green performance optimization design scenario text and photo data input by the designer, the temperature, humidity, air quality, and air velocity parameters are taken into account, taking into account the heat source distribution, personnel distribution, and usage scenario constraints, to understand the non-uniform indoor environment optimization design problems and respond to the typical problems of non-uniform environment optimization design of tall buildings;

[0040] In step S41, based on the end-cloud collaborative distributed work architecture, for complex non-uniform indoor environment green performance design decision problems, the data processing capability of the enhanced base model enabled by cloud knowledge representation is used to obtain a comprehensive understanding of the multimodal performance data fusion characteristics; in collaboration with a variety of architectural design and building simulation software or platforms, green performance optimization design decisions for non-uniform indoor environments of tall and large space buildings are formulated to get rid of the limitations of specific algorithms or software tools in terms of computing efficiency or computing cost;

[0041] In step S41, when the designer inputs optimization design task instructions into the big model, the cloud-side model understands the user needs and sends simulation instructions to the end-side model to guide the end-side model to perform simulation; the end-side model combines the cloud-side instructions with the engineering scenario model or non-uniform indoor environment parameters to perform green performance simulation optimization, output the simulation results, and provide a decision-making basis for the cloud-side model.

[0042] Furthermore, in step S42, a digital twin model of the large-space building is constructed; cloud computing is used to analyze the parameter and energy consumption changes caused by the modification of the optimization scheme in real time, and combined with VR and AR technologies, a visual comparison of the optimization design scheme is achieved, and real-time collaborative adjustment of the optimization design scheme by multiple parties is supported.

[0043] The present invention also proposes an electronic device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of a method for optimizing the design and decision-making of the non-uniform indoor environment of a tall space building based on a multimodal knowledge graph enhanced base model are implemented.

[0044] The present invention also proposes a computer-readable storage medium for storing computer instructions, which, when executed by a processor, implement the steps of a method for optimizing the design of non-uniform indoor environments of tall and large-space buildings based on a multimodal knowledge graph-enhanced base model.

[0045] Beneficial effects of the present invention:

[0046] In response to the complexity of the non-uniform environment of tall and large space buildings, the multimodal characteristics of data (such as numerical values, text, images, videos, etc.) are emphasized, and a method for optimizing the design and decision-making of the non-uniform indoor environment of tall and large space buildings based on a multimodal knowledge graph-enhanced base model is proposed. Through the knowledge representation of the multimodal data of the non-uniform indoor environment of tall and large space buildings, combined with the specific application engineering scenario types, a multimodal knowledge graph and vector database of green performance data of non-uniform indoor environment are constructed to effectively represent and store multimodal data. In the design decision-making process, a green performance optimization design model that fits the typical engineering scenario of non-uniform indoor environment of tall and large space buildings is constructed to understand the optimization design problem of non-uniform indoor environment, and a method for optimizing design decisions is proposed to output numerical values ​​such as building morphological parameters, spatial parameters, HVAC system parameters, and green performance image data of typical sections of tall and large space buildings. On this basis, a method for constructing an enhanced base model empowered by knowledge representation is proposed. Based on the end-cloud collaborative distributed work architecture, the optimal allocation of computing resources is achieved, and the collaborative multimodal knowledge graph and vector database are used to improve the efficiency of knowledge retrieval and question-answering, reduce computing costs and time, and improve computing accuracy. Combined with the digital sandbox, the visual interaction of the optimized design solution is realized. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0048] Figure 1 This is a flow chart of a method for optimizing the design of non-uniform indoor environments of tall and large-space buildings based on a multimodal knowledge graph-enhanced base model as described in the present invention.

[0049] Figure 2 This is a block diagram of the method for optimizing the design of non-uniform indoor environments of tall and large space buildings based on a multimodal knowledge graph-enhanced base model described in the present invention. DETAILED DESCRIPTION

[0050] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0051] Combination Figure 1-Figure 2 The present invention proposes a non-uniform indoor environment optimization design decision method for tall and large space buildings based on a multimodal knowledge graph enhanced base model, the method comprising the following steps:

[0052] S1. Acquire knowledge of multimodal data of non-uniform indoor environments; comprehensively apply LoRa-based MEMS thermal environment monitoring sensor arrays fixed on building facades or installed on smart cleaning or service robots, thermal imaging cameras and multi-wavelength high-density point cloud LiDARs carried on drones and other indoor environment physical parameter acquisition equipment and high-resolution cameras, multi-spectral imaging quantum dot infrared imagers, ultra-wideband (UWB) multiple-input multiple-output (MIMO) radars, flexible high-speed organic light detection devices and other user data acquisition equipment to collect and model the multimodal data of non-uniform indoor environments of tall and large space buildings.

[0053] The step S1 comprises the following steps:

[0054] S11, collect green performance data of non-uniform indoor environment; in the step S11, based on the dynamic acquisition of four types of data information of non-uniform indoor environment values, text, pictures, and videos on integrated platforms such as BIM and IoT, a green performance data set for non-uniform indoor environment is constructed, and the data modality type and corresponding data format requirements in the data analysis process are determined. BIM platform data is obtained in IFC format, and IoT platform data is obtained in JSON format. Numerical data is input in CSV, JSON and other formats, text data is input in TXT, CSV, JSON and other formats, image data is input in JPEG, PNG and other formats, and video data is input in MP4, AVI and other formats.

[0055] S12. Generate an embedded representation that integrates multimodal non-uniform indoor environment green performance information; use cross-modal alignment algorithms such as CLIP and MMV and multimodal Transformer to process building green performance data of different modalities, and convert these data into high-dimensional vectors that retain their multimodal data characteristics, embed them into a shared vector space to achieve information fusion and alignment, retain and strengthen the association between different modal building green performance data, such as the association between images and text descriptions, to assist in the understanding of the semantic relationship of multimodal data, so that the model can consider multiple modal information simultaneously when processing multimodal tasks.

[0056] S13, extracting green performance data information of non-uniform indoor environment; in the step S13, information extraction includes entity extraction, relationship extraction and event extraction; the entity extraction refers to applying named entity recognition (NER) method based on BiLSTM-CRF deep learning model, using context information, to extract specific building green performance objects or concepts from unstructured non-uniform indoor environment green performance data set; the relationship extraction refers to using OpenIE and BERT relation classification model or dependency syntax analysis method to identify the relationship between the above-mentioned non-uniform indoor environment green performance entities; the event extraction refers to the spatiotemporal dimension parameters of multiple non-uniform indoor environment green performance entities, which are detected by ACE event extraction framework, and the non-uniform indoor environment green performance event trigger words and related information are identified based on deep learning models such as BiLSTM-CRF and BERT.

[0057] S2. Integrate the multimodal data knowledge of non-uniform indoor environment to construct a multimodal knowledge graph and vector database of non-uniform indoor environment; apply the multidimensional knowledge graph to expand the knowledge representation of data such as temperature, humidity, noise level, air quality, air velocity, crowd distribution, passenger behavior, etc. of non-uniform indoor environment of tall space buildings, and combine with engineering scene types such as industrial plants, stadiums, airport terminals, and large convention and exhibition centers to construct a multimodal knowledge graph and vector database of green performance of non-uniform indoor environment of tall space buildings.

[0058] The step S2 comprises the following steps:

[0059] S21, converting the extracted information into RDF triples; in the step S21, the non-uniform indoor environment green performance data is converted into RDF triples of "entity-relationship-entity" through Python's RDF toolkit RDFLib, and the triples can be stored in a graph database for subsequent processing and query operations.

[0060] S22, construct a multimodal knowledge graph of non-uniform indoor environment based on RDF triples; in the step S22, by aligning the extracted entities with the existing entities of the non-uniform indoor environment, a relationship graph between entities is constructed, and the entities are linked using graph matching algorithms such as PageRank and Random Walk. For the records pointing to the same entity in the knowledge base of the non-uniform indoor environment, the distance calculation of the entities and relationships embedded in the vector space is merged by clustering algorithms such as K-means, DBSCAN or DeepWalk algorithm. Relation similarity calculation is performed using embedding models such as Word2Vec and GloVe, and the relationship between the same entities of different data sources is integrated using graph neural networks such as GCN and RGCNs. The integrated data is reasoned based on rules, according to the reliability of the data source, timestamp and other set priority rules, to eliminate conflicting or contradictory knowledge obtained from different data sources, and derive new knowledge; based on the reasoning of machine learning such as GNN, the implicit green performance data relationship of the non-uniform indoor environment is mined, knowledge is enhanced, and the accuracy of information extraction and the reasoning ability of the knowledge graph are improved.

[0061] S23. Determine the elements of typical non-uniform indoor environment engineering scenarios; for tall and large-space buildings with non-uniform indoor environment characteristics, such as industrial plants, stadiums, airport terminals, and large convention and exhibition centers, identify their engineering scenario types and extract corresponding green performance evaluation indicators, including power intensity, equipment energy consumption ratio, building energy consumption, indoor air quality, thermal comfort, lighting energy efficiency, etc.

[0062] S24, build and update the corresponding multimodal knowledge graph and vector database; in the step S24, pre-process the extracted non-uniform indoor environment multimodal data. Normalize and reduce the dimension of real-time monitoring numerical structured data such as temperature, humidity, air velocity, light intensity, and energy consumption; generate embedding vectors for text data such as project information, user needs, design specifications and standards, and policy documents based on natural language processing models such as BERT and TF-IDF; extract feature vectors from image data such as building photos, renderings, general plans, floor plans, sections, elevations, detail drawings, and pseudo-color images of optimized simulation results using ResNet and CNN models, and extract features from key frames using C3D networks and LSTM models for video data such as user behavior videos and building three-dimensional models. Based on the generated embedding vectors, standardize the feature dimensions of green performance data. Construct an HNSW vector index structure to improve the speed of vector similarity search, provide support for the matching of semi-structured data in the multimodal knowledge graph, and accelerate the intelligent question-and-answer process for user design problems. For the green performance evaluation indicators of the typical engineering scenarios, further determine the influencing factors, including equipment distribution, wall insulation performance, natural lighting, personnel flow, dynamic load, etc.; and clarify the typical models or algorithms in calculation or simulation, including CFD models, sunshine analysis models, dynamic energy consumption analysis models, PMV models, PPD models and energy consumption optimization algorithms, regional zoning control algorithms, etc. Construct a green performance portrait of the non-uniform indoor environment of a specific tall and large space engineering scenario, and complete and update the multimodal green performance database of a specific tall and large space engineering scenario.

[0063] S3. Strengthen the knowledge retrieval of optimization design decision-making problems, and construct a green performance optimization design model that fits the typical engineering scenarios of non-uniform indoor environments; based on the multimodal knowledge graph and vector database of non-uniform indoor environments, construct a green performance optimization design model for typical engineering scenarios of non-uniform indoor environments that coordinates the indoor environment physical parameter collection equipment, the user data collection equipment, HVAC equipment, intelligent operation and maintenance robots, edge computing equipment, and cloud computing platforms.

[0064] The step S3 comprises the following steps:

[0065] S31, modularly cluster the knowledge in the knowledge graph and obtain a structured subgraph; in step S31, the community detection algorithm is used to identify the interrelated node groups and divide the knowledge graph into different communities. The graph embedding algorithm is used to identify and add missing relationships or entities to enhance the integrity of the knowledge graph and assist knowledge reasoning.

[0066] S32, understand the design problem, and improve the knowledge retrieval efficiency based on the structured subgraph; in the step S32, based on the design problem text, real-life photos, plan and elevation section engineering drawings and other green performance optimization design engineering scene data input by the designer, understand the potential green performance hidden dangers of the specific optimization design scene, and take into account the indoor physical environment parameters such as temperature, humidity, illumination, air quality, air pollutant concentration, ventilation volume, equipment distribution, wall insulation performance, natural lighting, personnel flow, dynamic load and other limiting factors to understand the optimization design problems of non-uniform indoor environment. Through hierarchical clustering methods such as hierarchical clustering and k-means clustering, abstracts are extracted from typical engineering scenes of multi-dimensional non-uniform indoor environments, and communities are graded and analyzed. The structured subgraph and its abstract are converted into high-dimensional vector representations through algorithms such as Node2Vec, DeepWalk, and GraphSAGE. High-dimensional vector retrieval is assisted by algorithms such as FAISS and HNSW to improve the efficiency of machine learning tasks such as clustering and retrieval.

[0067] S33, construct a green performance optimization design model that fits the typical engineering scenario of non-uniform indoor environment; in the step S33, based on the multimodal knowledge graph that fits the typical engineering scenario of green performance optimization design of non-uniform indoor environment, combined with multimodal Transformer, GNN, RNN and other neural network models, the coupling relationship between multimodal data involved in the task of optimizing design of non-uniform indoor environment is processed to enable the traditional decision-making model based on the subjective experience of designers. In the step S33, the main output data includes numerical and image data to provide design alternatives at the levels of building form, spatial layout, energy system design, etc., respond to the climate, environment and cultural requirements of the design task, and assist in optimizing the design decision. In the step S33, the model is used to coordinate a variety of architectural design and building simulation software or platforms to process multimodal building green performance data, and get rid of the limitations of specific algorithms or software tools in computing efficiency or computing cost. In step S33, the output numerical data include building form parameters such as building shape coefficient and window-to-wall ratio, building space parameters such as span, depth, and floor height, HVAC system parameters such as equipment operating time and power consumption, etc.; the output image data include temperature gradient prediction maps of large-space buildings, flow field prediction maps, glare simulation maps, and other green performance predictions for typical sections of large-space buildings.

[0068] The green performance optimization design model for the typical engineering scenario of the non-uniform indoor environment includes:

[0069] Monitoring module: including physical parameter acquisition submodule and user data acquisition submodule. The physical parameter acquisition submodule includes: a LoRa-based MEMS thermal environment monitoring sensor array fixed on the building facade or installed on an intelligent cleaning or service robot to collect data such as non-uniform indoor environment temperature, humidity, air quality, air flow rate, etc. of tall and large space buildings; a pipeline detection robot, which uses television imaging to detect cracks, blockages, corrosion, and other problems in HVAC equipment pipelines; a high-performance wheeled robot to detect abnormal conditions such as fire and smoke and monitor indoor safety; and a drone to monitor and analyze the thermal distribution of the facades of tall buildings. The user data acquisition submodule includes: a high-resolution camera to identify the movement of people based on computer vision algorithms; a quantum dot infrared imager with multi-spectral imaging to monitor the density of people in the space in real time; an ultra-wideband (UWB) multiple-input multiple-output (MIMO) radar to obtain high-quality images of breathing signals of multiple people in scenes such as entrance halls, security check areas, waiting areas (cars), and commercial areas; and a flexible high-speed organic light detection device to collect physiological signals such as breathing, heart rate, and skin electricity of users in the office area.

[0070] Retrieval module: Based on cross-modal alignment technology, using the data processing capabilities of the cloud platform, obtain a comprehensive understanding of the fusion characteristics of multimodal performance data and design problems. For the design problems input by the user, based on the "knowledge retrieval" and "retrieval enhancement generation" technologies, the design problems are vectorized, and similarity retrieval and comparison are performed with the multimodal knowledge graph and vector database. The knowledge graph is used to structure semantic problems and determine the relationship between feature vectors and entities, relationships, and attributes in the graph database; the vector database is used to store data feature vectors to find the most relevant matches for green performance multimodal data. Based on knowledge such as similar cases at the semantic level, the ability to capture key design issues is enhanced, computational simulation instructions are generated, and the structured subgraphs of the knowledge graph involved in the optimization design problem are clarified.

[0071] Computing module: Collaborating with the retrieval module, based on the computing simulation instructions and the structured subgraph of the knowledge graph, integrating the element features of the typical engineering scenarios of the tall and large space buildings, and utilizing the lightweight and personalized models deployed on the end side to realize layered lightweight decision-making, dynamically analyzing the green performance evaluation indicators of the non-uniform indoor environment, and integrating multimodal data on the basis of data collection, combining with thermal simulation software to simulate heat transfer, hot air flow, light intensity, etc. in the building, as well as CFD simulation of air flow patterns, heat distribution and temperature gradients, responding to indoor environments in different seasons and different usage scenarios, analyzing and formulating optimization design decisions such as air conditioning design, ventilation system layout, and heat source distribution, and outputting the architectural form, space, and HVAC system design corresponding to the decisions.

[0072] Interaction module: Combine the output results of the calculation module to build the digital twin model of the tall space building. Combined with VR and AR technology, the decision-making support data images of the typical horizontal and vertical sections, such as temperature, humidity, air quality, air velocity, crowd distribution, and passenger behavior, are projected onto the corresponding model surface. Designers manipulate the digital sandbox by wearing VR equipment or moving the equipment components in the physical model; the cloud platform calculates and analyzes the changes in parameters and energy consumption caused by the operation in real time, and marks the recommended layout positions of the equipment in the alternative optimization design scheme for the designer in the digital twin model, so as to realize the visual comparison of the optimization design scheme and support the real-time collaborative adjustment of the optimization design scheme by multiple personnel.

[0073] The digital twin model of the large-scale building includes: outputting numerical and image data such as non-uniform indoor environmental temperature, humidity, air quality, air flow rate, etc. of the large-scale building through the MEMS thermal environment monitoring sensor array; outputting crowd density image data through the infrared imager; outputting building form parameters such as building shape coefficient and window-to-wall ratio, and numerical data such as building space parameters such as bay, depth, and floor height through LiDAR; outputting image data such as temperature gradient prediction diagrams and flow field prediction diagrams of large-scale buildings for green performance predictions of typical sections of large-scale buildings through thermal simulation and CFD simulation; and data such as the operating time and power consumption of the HVAC equipment.

[0074] S4. Construct a non-uniform indoor environment design decision cloud platform model, coordinate the cloud-side and terminal-side models, and flexibly handle optimization design problems; based on the non-uniform indoor environment optimization design model, integrate the flat, vertical and cross-sectional wind field diagrams of large-space buildings collected by the MEMS thermal environment monitoring sensor array, the thermal map generated by the infrared imager capturing thermal radiation, the three-dimensional model of large-space buildings generated based on the high-precision spatial data of LiDAR, thermal simulation software and CFD simulation results, and build a visual digital twin model of the non-uniform indoor environment performance of large-space buildings based on VR and AR technology; for the HVAC equipment layout plan of typical functional spaces such as entrance halls, security check areas, waiting (car) areas, commercial areas, and office areas, realize the designer's visual interaction in the form of a digital sandbox.

[0075] The step S4 comprises the following steps:

[0076] S41. Build an end-cloud collaborative architecture that is suitable for typical engineering scenarios of various non-uniform indoor environments.

[0077] In step S41, based on the basic data of the green performance optimization design scene such as text and photos input by the designer, parameters such as temperature, humidity, air quality, air flow rate, etc. are taken into account, taking into account limiting factors such as heat source distribution, personnel distribution, and usage scenarios, to understand the problems of non-uniform indoor environment optimization design and respond to typical problems of non-uniform environment optimization design of tall buildings.

[0078] Based on the end-cloud collaborative distributed work architecture, for complex green performance design decision problems in non-uniform indoor environments, the data processing capabilities of the enhanced base model enabled by cloud knowledge representation are used to obtain a comprehensive understanding of the multi-modal performance data fusion characteristics. Collaborating with a variety of architectural design and building simulation software or platforms, green performance optimization design decisions for non-uniform indoor environments in tall and large-space buildings are made to get rid of the limitations of specific algorithms or software tools in terms of computational efficiency or computational cost.

[0079] When the designer inputs optimization design task instructions into the big model, the cloud-side model understands the user needs and sends simulation instructions to the device-side model to guide the device-side model to perform simulation; the device-side model combines the cloud-side instructions with the engineering scenario model or non-uniform indoor environment parameters to perform green performance simulation optimization, providing the cloud-side model with simulation results and other decision-making basis.

[0080] S42. Build a digital twin model of a tall and large-space building, and display the optimized design results through a digital sandbox to achieve visual interaction.

[0081] In step S42, a digital twin model of the large-space building is constructed; cloud computing is used to analyze the parameter and energy consumption changes caused by the modification of the optimization scheme in real time, and VR and AR technologies are combined to achieve a visual comparison of the optimization design scheme, and support real-time collaborative adjustment of the optimization design scheme by multiple parties.

[0082] For tall and large-space buildings with non-uniform indoor environment characteristics, such as industrial plants, stadiums, airport terminals, and large exhibition centers, the data of the hardware equipment of the MEMS thermal environment monitoring sensor array, camera, and infrared imager based on LoRa are integrated. By analyzing the trend of sensor data changes over time and the peak hours of personnel flow, the time series and spatial characteristics of the engineering scene are analyzed. Based on the deep learning model, historical data are analyzed to identify and classify the types of engineering scenes, extract green performance evaluation indicators, determine its influencing factors and elements such as typical models or algorithms in calculation or simulation, complete and update the multimodal green performance database, and store it in the graph database. Based on the deep learning algorithm for processing graph structure data, the interconnected green performance data are traversed to establish a unified representation method for cross-scale multimodal information such as text, raster images, three-dimensional models, plane vectors, and building performance in the field of architecture, and the input discrete data is converted into a high-dimensional vector in a unified multimodal shared vector space. According to the engineering scene, a multimodal knowledge graph that fits the typical engineering scene of green performance optimization design for non-uniform indoor environment is constructed. By generating embedded vectors, standardizing the feature dimensions of green performance data, constructing an HNSW vector index structure, improving the vector similarity search speed, and building a green performance data vector database for non-uniform indoor environments, support is provided for the matching of semi-structured data in multimodal knowledge graphs.

[0083] The indoor environment monitoring sensor array includes: a resistive flexible temperature sensor, a capacitive flexible humidity sensor, a CO monitoring sensor 2 , CO, methane and other gas concentrations; optical particle counters for monitoring suspended particles such as PM2.5 and PM10 in the air; and particle image velocimeters for monitoring air flow rate.

[0084] The knowledge graph and vector database support multimodal data input including numerical values ​​(including indoor environment data, building form data, green performance data, etc.), text (project information, user needs, design specifications and standards, policy documents, etc.), images (building photos, renderings, master plans, floor plans, sections, elevations, detailed drawings, pseudo-color images of optimization simulation results, etc.), and videos (user behavior videos, building three-dimensional models, etc.).

[0085] The green performance evaluation indicators include pollutant concentration, time and area ratio of indoor environmental parameters in the adaptive thermal comfort zone under natural ventilation or combined ventilation conditions of main functional rooms, temperature range of inner surface of exterior walls and roofs, average natural ventilation air changes in main functional rooms under typical conditions in transition seasons, energy consumption of cold and heat sources, energy consumption of transmission and distribution systems, utilization rate of renewable energy, electricity intensity, etc.

[0086] Typical problems in the optimization design of indoor environment of tall buildings include: uneven distribution of longitudinal heat sources in tall space buildings, which requires balancing energy consumption and comfort, analyzing heat convection movement, and determining local heating and fresh air system solutions; large temperature and humidity gradients caused by height differences, which requires coordinating temperature and humidity and formulating air conditioning system zoning design plans; uneven air flow, which requires determining air stagnation areas, installing air circulation fans, combining natural ventilation and mechanical ventilation, and overcoming spatial height differences and airflow stratification problems.

[0087] Example

[0088] like Figure 1 As shown, a non-uniform indoor environment optimization design decision method based on a multimodal knowledge graph enhanced base model involved in this embodiment is targeted at large space buildings such as terminal buildings. The indoor environment of the terminal building is complex, the flow of people is unevenly distributed, some areas are densely populated, functions are alternating, and the frequency of use and demand vary greatly. The temperature and humidity of the indoor environment are unevenly distributed, and heat accumulation and poor air flow are prone to problems. In the design, it is necessary to balance the needs of energy saving and comfort on the basis of ensuring the passenger flow, and realize intelligent control.

[0089] S1, obtain multimodal data knowledge of non-uniform indoor environment;

[0090] In this embodiment, data collection is based on BIM and IoT platforms. The data collected based on the BIM platform include: geometric spatial data such as building size, location, spatial layout, window-to-wall ratio, thermal conductivity and thermal resistance of building materials, acoustic properties and other material physical properties, basic parameters and pipeline layout of building HVAC systems and other electrical equipment, energy monitoring data such as lighting and energy consumption, equipment maintenance records, operation schedules and other building operation and maintenance data. Based on the IoT platform, real-time dynamic data of the terminal building can be obtained, including: indoor physical environment parameters such as temperature and humidity, air quality, noise level, and illumination of the working surface, energy consumption monitoring data such as real-time total energy consumption of HVAC system equipment or real-time energy consumption of specific areas and specific equipment, user-related data such as crowd distribution and passenger behavior, etc. At the same time, it is possible to record the dynamic changes of outdoor physical environment parameters such as meteorological data, wind speed and direction, and supplement project information, user needs, design specifications and standards, policy documents, etc.

[0091] The geometric space data includes parameter numerical data, descriptive text data and image data such as architectural photos, renderings, master plans, floor plans, sections, elevations, and detail drawings; the material physical properties include numerical data and descriptive text data; the energy monitoring data include numerical data with time series information, descriptive text data, pseudo-color images and other image data and video data; the indoor physical environment data include numerical data with time series information, pseudo-color images and other image data; the user-related data include questionnaire text data, photos and other image data, behavior recordings and other video data.

[0092] Preprocess the raw data from the different data sources to extract structured information. For numerical data in the form of tables, the above processing is not required. For text data, semantic segmentation, part-of-speech tagging, etc. are implemented based on the NLP or LLM model.

[0093] Using the multimodal Transformer deep learning architecture, using models such as ViLBERT and UNITER, we can embed data of different modalities, such as numbers, text, images, and videos, into vector space through different encoders. We can use the self-attention mechanism to distribute weights between different modalities, enhance the understanding of the association between data of different modalities, such as the understanding of the relationship between the terminal profile and the corresponding annotation text; generate representations related to both modalities, and retain the temporal information of dynamic data.

[0094] For the preprocessed data, extract the entity and relationship data therein. Apply deep learning-based models such as BERT, RoBERTa, and spaCy to classify and identify entities, which contain category information related to the terminal's green performance objects or concepts, and constitute the subject or object of the RDF triple. Use OpenIE, BERT, and other relational classification models or dependency syntactic analysis methods to identify the relationship between the above-mentioned terminal indoor environment green performance entities and output the predicate of the RDF triple.

[0095] For event data involving multiple entities, attributes, and time series information, the ACE event extraction framework is used to mark entities and relationships, and deep learning models such as BiLSTM-CRF and BERT are used to identify event trigger words, roles, and other information, and convert them into triples.

[0096] S2, integrate the multimodal data knowledge of non-uniform indoor environment and construct the multimodal knowledge graph and vector database of non-uniform indoor environment;

[0097] The green performance data of the terminal's indoor environment is converted into RDF triples of "entity-relationship-entity" through Python's RDF toolkit RDFLib and stored in the Neo4j graph database.

[0098] When new knowledge about the green performance of the indoor environment of the terminal is extracted, the above steps are repeated to extract the entity, relationship, and attribute information of the knowledge, and the newly extracted green performance entity of the indoor environment of the terminal is identified using context information, and is labeled, embedded in the vector space, and linked based on semantic similarity calculation. For text information, entities can be matched based on the similarity of strings. For records pointing to the same entity in the graph database, based on clustering algorithms such as K-means and DBSCAN or machine learning methods such as logistic regression and random forest, items with similar names or the same ID are screened, and whether the relationships of different inputs are the same is determined by pattern matching or rule reasoning; using graph embedding methods such as DeepWalk and Node2Vec, the newly extracted green performance entity of the indoor environment of the terminal and its relationships and attributes are embedded in the vector space, and duplicate entities are identified and merged through distance calculation. On this basis, the semantic similarity calculation is used to determine whether the relationships between entities are consistent, and similar relationships from different sources are fused through graph convolutional neural networks or relationship attention networks. In the case of conflicts in entities or attributes, the data is sorted based on the reliability of the terminal data source, timestamp, value range and other factors, and the data with the highest priority is selected as the credible data. The ontology reasoning algorithm is used to verify that the logical constraints between the data meet the objective laws and architectural common sense. On this basis, the GNN algorithm is used to further infer new knowledge.

[0099] For the terminal project scenario, in view of its complex indoor environment, uneven distribution of passenger flow and changes over time, the evaluation indicators include: temperature, relative humidity, air flow rate, air quality (carbon dioxide concentration, PM2.5 concentration) in different indoor areas, height surfaces and typical profiles, illumination of the working surface, power consumption intensity of HVAC systems and lighting systems in different areas, energy consumption of equipment, thermal comfort of users, changes in passenger density, etc.

[0100] The distribution of cold and heat loads in the terminal space is complex. Affected by its large-scale depth, bay, and height, different areas have different influencing factors. The factors to be considered include: 1. Whether it is close to the curtain wall. The area close to the curtain wall is greatly affected by factors such as outdoor solar radiation and infiltration wind. The area far from the curtain wall is mainly affected by personnel mobility and density, and equipment heat dissipation; 2. Net height. Due to the floating of hot air, the temperature difference between the upper and lower areas of the space is large; 3. Areas with different usage modes. The main areas of the terminal include the entrance hall, check-in area, security check area, waiting area, baggage claim area, commercial area, etc. Among them, the entrance hall has strong personnel mobility, the check-in area has the characteristics of high-density and short-term gathering of personnel, the security check area and the baggage claim area have two modes of personnel retention and flow, the waiting area has a long retention time, and the commercial area has a large fluctuation in personnel density.

[0101] Preprocess the data, normalize and reduce the dimension of real-time monitoring numerical structured data such as temperature, humidity, air velocity, light intensity, energy consumption, etc. in different areas or heights and different usage scenarios, and extract the feature vectors of image data such as pseudo-color images of typical regional profile optimization simulation results and user behavior videos. Perform natural language processing on the common design specifications and standards, policy documents and other text data of the terminal to generate embedded vectors; extract feature vectors from specific terminal building photos, renderings, general plans, floor plans, section drawings, elevations, detailed large drawings and other image data using ResNet and CNN models, and extract features from key frames for the three-dimensional model of the building using C3D network and LSTM model. On this basis, construct the HNSW vector index structure, construct the vector database of the terminal indoor environment multimodal knowledge graph, and assist in the green performance optimization design decision-making of the non-uniform indoor environment of tall and large space buildings, such as uneven longitudinal heat source distribution, large temperature and humidity gradients, uneven air flow, local glare or insufficient lighting.

[0102] S3. Strengthen the knowledge retrieval of optimization design decision-making problems and build a green performance optimization design model that fits typical engineering scenarios of non-uniform indoor environments;

[0103] For the terminal green performance data knowledge graph, Louvain, Label Propagation and other community detection algorithms are used to perform rough division, effectively segmenting the community structure in the larger knowledge graph. Node2Vec and DeepWalk random walks are used to embed text nodes and graph nodes respectively, to enhance the integrity of the knowledge graph and assist knowledge reasoning.

[0104] Hierarchical clustering methods such as hierarchical clustering and k-means clustering are used to extract the terminal engineering scene summary. The entities and relationships of the terminal green performance knowledge are converted into high-dimensional vectors using Node2Vec, DeepWalk, and GraphSAGE algorithms. The embedded communities and entities are stored in a vector database using FAISS, HNSW and other algorithms to assist retrieval.

[0105] Based on the design problem text input during the question-and-answer process, real-life photos of specific terminal scenes, terminal plan, elevation and section engineering drawings, typical terminal section simulation drawings and other green performance optimization design engineering scenario data, determine the specific optimization design scene type (entrance hall, check-in area, security area, waiting area, baggage claim area, commercial area), understand the potential green performance hidden dangers of the specific optimization design scene, search for the regional plane and typical section temperature, relative humidity, air flow rate, air quality (carbon dioxide concentration, PM2.5 concentration) distribution related to the problem, regional HVAC system, lighting system power intensity, equipment energy consumption, as well as users' thermal comfort, changes in crowd density, etc. For the check-in area and security area, it is necessary to take into account the illumination information of the working surface.

[0106] The CFD model, sunshine analysis model, dynamic energy consumption analysis model, PMV model, PPD model, energy consumption optimization algorithm, regional zoning control algorithm, etc. are collaboratively called to handle specific green performance simulation analysis problems, and the outdoor physical environment elements of the site and the indoor physical environment data of multiple prediction models are coupled to support the decision-making of terminal green performance issues and assist designers in the selection of solutions. The terminal local form or spatial layout optimization opinions, energy system or technical platform optimization design opinions and other alternative solutions are output, as well as the terminal body coefficient or window-to-wall ratio and other architectural form parameters that support these data, the specific area bay or depth and other architectural space parameters after optimizing the spatial layout, the regional equipment operating time of the existing or multiple optimization solutions, the power consumption and other HVAC system parameters, the typical section building temperature gradient prediction map, flow field prediction map, glare simulation map, etc., to get rid of the limitations of specific algorithms or software tools in terms of computational efficiency or computational cost.

[0107] S4. Build a non-uniform indoor environment design decision cloud platform model, coordinate cloud-side and device-side models, and flexibly handle optimization design problems;

[0108] Based on the data set obtained by executing design simulation, decision-making and other tasks using the green performance optimization design model that fits the typical engineering scenario of the terminal, the computing power and data processing capabilities of the cloud platform are brought into play. Based on the Mixtral8x78 large model, an enhanced base model empowered by the green performance knowledge representation of the terminal is constructed to fully understand the multimodal building information involved in the terminal scenario, and better support the reasoning and analysis capabilities of the green performance optimization design model to deal with complex problems.

[0109] On this basis, the collaborative analysis capabilities of the cloud-side enhanced base model and the end-side green performance optimization design model are strengthened. Reduce computing costs and time, and improve computing efficiency and accuracy. When the designer inputs optimization design task instructions to the large model, the cloud-side model understands the user's needs and issues simulation instructions to the end-side model to guide the end-side model to simulate; the end-side model combines the cloud-side instructions with the engineering scenario model or non-uniform indoor environment parameters to perform green performance simulation optimization, output simulation results, and provide decision-making basis for the cloud-side model.

[0110] For the green performance data of the indoor environment of the terminal building that is updated in real time by IoT, the data is collected in real time through IoT sensors. According to the requirements of the sampling and measurement accuracy, the analysis interval is determined, data analysis is performed, and corresponding decision-making reference plans are provided. Designers or terminal operation and maintenance personnel compare or modify the alternative plans and formulate operation plans for indoor physical environment control equipment. On the basis of excluding abnormal data, the statistical parameters such as the mean value and standard deviation of the data in different time series are analyzed to summarize the phased regular characteristics; when the deviation exceeds the preset critical value, the knowledge graph is updated.

[0111] In the process of solving the green performance optimization design, a green performance portrait of the terminal indoor environment is constructed, and the terminal multimodal green performance database is completed and updated.

[0112] The present invention also proposes an electronic device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of a method for optimizing the design and decision-making of the non-uniform indoor environment of a tall space building based on a multimodal knowledge graph enhanced base model are implemented.

[0113] The present invention also proposes a computer-readable storage medium for storing computer instructions, which, when executed by a processor, implement the steps of a method for optimizing the design of non-uniform indoor environments of tall and large-space buildings based on a multimodal knowledge graph-enhanced base model.

[0114] The memory in the embodiments of the present application may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), and direct RAM bus RAM (DR RAM). It should be noted that the memory of the method described in the present invention is intended to include, but is not limited to, these and any other suitable types of memory.

[0115] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (digital subscriber line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a high-density digital video disc (DVD)), or a semiconductor medium (eg, a solid state disc (SSD)).

[0116] In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in a processor or an instruction in the form of software. The steps of the method disclosed in conjunction with the embodiment of the present application can be directly embodied as a hardware processor for execution, or a combination of hardware and software modules in a processor for execution. The software module can be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in a memory, and the processor reads the information in the memory and completes the steps of the above method in conjunction with its hardware. To avoid repetition, it is not described in detail here.

[0117] It should be noted that the processor in the embodiment of the present application can be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method embodiment can be completed by an integrated logic circuit of hardware in the processor or an instruction in the form of software. The above processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiment of the present application can be directly embodied as a hardware decoding processor to perform, or the hardware and software modules in the decoding processor can be combined and performed. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in a memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0118] The above is a detailed introduction to the method for optimizing the design and decision-making of the non-uniform indoor environment of large-space buildings based on a multimodal knowledge graph enhanced base model proposed in the present invention. This article uses specific examples to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for general technical personnel in this field, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A method for optimizing the design of non-uniform indoor environment of tall and large space buildings based on a multimodal knowledge graph-enhanced base model, characterized in that: The method comprises the following steps: S1, obtain multimodal data knowledge of non-uniform indoor environment; The step S1 comprises the following steps: S11. Collect green performance data of non-uniform indoor environment; S12, generating an embedded representation of the green performance information of the fused multi-modal non-uniform indoor environment; S13, extracting green performance data information of non-uniform indoor environment; S2, integrate the multimodal data knowledge of non-uniform indoor environment and construct the multimodal knowledge graph and vector database of non-uniform indoor environment; The step S2 comprises the following steps: S21, converting the extracted information into RDF triples; S22, constructing a multimodal knowledge graph for non-uniform indoor environments based on RDF triples; S23. Determine the elements of typical non-uniform indoor environment engineering scenes; S24. Build and update the corresponding multimodal knowledge graph and vector database; S3. Strengthen the knowledge retrieval of optimization design decision-making problems and build a green performance optimization design model that fits typical engineering scenarios of non-uniform indoor environments; The step S3 comprises the following steps: S31. Modularize the knowledge in the knowledge graph and obtain a structured subgraph; S32, understand design problems and improve knowledge retrieval efficiency based on structured subgraphs; S33. Construct a green performance optimization design model that fits typical engineering scenarios of non-uniform indoor environments; S4, end-cloud collaboration, flexibly handles optimization design problems, and realizes visual interaction of optimization design results in the form of digital sandbox; The step S4 comprises the following steps: S41. Build a terminal-cloud collaborative architecture that meets typical engineering scenarios of various non-uniform indoor environments; S42. Build a digital twin model of a tall and large-space building, and display the optimized design results through a digital sandbox to achieve visual interaction.

2. The method according to claim 1, characterized in that In the step S13, information extraction includes entity extraction, relationship extraction and event extraction; the entity extraction refers to the application of named entity recognition method based on BiLSTM-CRF deep learning model, using context information to extract specific building green performance objects or concepts from unstructured non-uniform indoor environment green performance data set; the relationship extraction refers to the use of OpenIE and BERT relational classification model or dependency syntax analysis method to identify the relationship between the above-mentioned non-uniform indoor environment green performance entities; the event extraction refers to the spatiotemporal dimension parameters of multiple non-uniform indoor environment green performance entities, which are detected by ACE event extraction framework, and the non-uniform indoor environment green performance event trigger words and related information are identified based on the deep learning model.

3. The method according to claim 1, characterized in that In the step S21, the non-uniform indoor environment green performance data is converted into RDF triples of "entity-relationship-entity" through Python's RDF toolkit RDFLib, and the triples are stored in a graph database for subsequent processing and query operations; In the step S22, the extracted entities are aligned with the existing entities in the non-uniform indoor environment to construct a relationship graph between the entities, and the entities are linked using a graph matching algorithm; for the records pointing to the unified entity in the non-uniform indoor environment knowledge base, the distance calculation of the entities and relationships embedded in the vector space is merged by a clustering algorithm or a DeepWalk algorithm; the relationship similarity is calculated using an embedding model, and the relationship between the same entities in different data sources is integrated using a graph neural network; The integrated data resolves conflicting or contradictory knowledge obtained from different data sources and derives new knowledge through rule-based reasoning according to priority rules set based on the reliability and timestamp of the data source; Based on machine learning reasoning, the implicit green performance data relationships of non-uniform indoor environments are mined to perform knowledge enhancement, thereby improving the accuracy of information extraction and the reasoning ability of knowledge graphs.

4. The method according to claim 1, characterized in that: In the step S24, the extracted non-uniform indoor environment multimodal data is preprocessed; the temperature, humidity, air velocity, light intensity and energy consumption real-time monitoring numerical structured data are normalized and dimensionally reduced; Generate embedding vectors based on natural language processing models for project information, user requirements, design specifications and standards, and policy document text data; For architectural photos, renderings, master plans, floor plans, sections, elevations, detail drawings, and pseudo-color images of optimized simulation results, ResNet or CNN models are used to extract feature vectors. For user behavior videos and architectural 3D model videos, C3D networks and LSTM models are used to extract features from key frames. In step S24, the green performance data feature dimension is standardized based on the generated embedding vector; Construct HNSW vector index structure to improve the speed of vector similarity search, provide support for matching semi-structured data in multimodal knowledge graphs, and accelerate the intelligent question-answering process for user design questions; In step S24, for the green performance evaluation indicators of typical engineering scenarios, further determine the influencing factors, including equipment distribution, wall insulation performance, natural lighting, personnel flow and dynamic load; and clarify the typical models or algorithm elements in the calculation or simulation, including CFD model, sunshine analysis model, dynamic energy consumption analysis model, PMV model, PPD model and energy consumption optimization algorithm, regional zoning control algorithm; In step S24, a green performance portrait of the non-uniform indoor environment of a specific large-space engineering scene is constructed, and a multimodal green performance database of the specific large-space engineering scene is completed and updated.

5. The method according to claim 1, characterized in that In step S31, the community detection algorithm is used to identify the interconnected node groups and divide the knowledge graph into different communities; the graph embedding algorithm is used to identify and add missing relationships or entities to enhance the integrity of the knowledge graph and assist knowledge reasoning.

6. The method according to claim 1, characterized in that In step S33, based on the multimodal knowledge graph that fits the typical engineering scenarios of green performance optimization design of non-uniform indoor environment, on the cloud platform, based on the base model, an enhanced base model enabled by the green performance knowledge representation of non-uniform indoor environment is trained to give full play to the advantages of cloud computing power, process the coupling relationship between multimodal data involved in the task of optimizing the design of non-uniform indoor environment, and enable the decision-making model based on the subjective experience of the designer.

7. The method according to claim 1, characterized in that In step S41, based on the green performance optimization design scenario text and photo data input by the designer, the temperature, humidity, air quality, air velocity parameters are taken into account, the heat source distribution, the personnel distribution, and the use scenario restriction factors are taken into account, the non-uniform indoor environment optimization design problems are understood, and the typical problems of the non-uniform environment optimization design of tall buildings are responded to; In step S41, based on the end-cloud collaborative distributed working architecture, for the complex non-uniform indoor environment green performance design decision problem, the data processing capability of the enhanced base model enabled by cloud knowledge representation is used to obtain a comprehensive understanding of the multimodal performance data fusion characteristics; Collaborate with a variety of architectural design and building simulation software or platforms to make green performance optimization design decisions for non-uniform indoor environments of tall and large-space buildings, and get rid of the limitations of specific algorithms or software tools in terms of computational efficiency or computational cost; In step S41, when the designer inputs optimization design task instructions into the big model, the cloud-side model understands the user needs and sends simulation instructions to the end-side model to guide the end-side model to perform simulation; the end-side model combines the cloud-side instructions with the engineering scenario model or non-uniform indoor environment parameters to perform green performance simulation optimization, output the simulation results, and provide a decision-making basis for the cloud-side model.

8. The method according to claim 1, characterized in that In step S42, a digital twin model of the large-space building is constructed; cloud computing is used to analyze the parameter and energy consumption changes caused by the modification of the optimization scheme in real time, and VR and AR technologies are combined to achieve a visual comparison of the optimization design scheme, and support real-time collaborative adjustment of the optimization design scheme by multiple parties.

9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.

10. A computer-readable storage medium for storing computer instructions, characterized in that: When the computer instructions are executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images