Multi-modal data fusion and analysis building operation and maintenance system based on BIM

Abstract

The invention discloses a BIM-based multi-modal data fusion and analysis building operation and maintenance system, which comprises a multi-modal data acquisition layer, a data fusion processing layer, an analysis decision-making layer and an operation and maintenance execution feedback layer, and is characterized in that static information and real-time multi-modal data of a building are deeply fused, spatial features and semantic information of a BIM model can be fully utilized, and the operation and maintenance performance of the BIM model is improved. According to the method, the operation state of the building is reflected more comprehensively and accurately to improve the operation and maintenance reliability, and meanwhile, fault prediction, energy consumption optimization and spatial layout adjustment can be performed on the building through analysis of the operation and maintenance data set, and visualization is performed in the BIM to improve the efficiency of building operation and maintenance.

Description

Technical Field

[0001] The present invention relates to the technical field of construction engineering, and particularly to a building operation and maintenance system based on BIM. Background Art

[0002] In traditional building operation and maintenance systems, the data acquisition channels are single, unable to comprehensively and deeply reflect the complex operating conditions inside the building, and each data exists independently, unable to be associated and comprehensively analyzed for the operation of the building. At the same time, traditional building operation and maintenance systems can only observe the trends of building operation and maintenance data and judge thresholds, resulting in lag in the operation and maintenance of buildings.

[0003] Existing building operation and maintenance systems based on Building Information Modeling (BIM), such as a maintenance system based on BIM technology disclosed in a Chinese patent with publication number CN 117633962 A, establish a basic model with BIM technology and add an operation and maintenance reference database, an operation log module, and an operation and maintenance warning module to the basic model. Through the display and combination of operation and maintenance data by the BIM model, visual management of operation and maintenance is achieved, resulting in low efficiency and reliability of building operation and maintenance.

[0004] Therefore, how to comprehensively collect, integrate, analyze, and utilize building operation and maintenance data based on BIM to effectively improve the efficiency and reliability of building operation and maintenance has become an urgent problem to be solved in this field. Summary of the Invention

[0005] Aiming at the defects of the prior art, the purpose of the present invention is to provide a building operation and maintenance system for multi-modal data fusion and analysis based on BIM, which can effectively improve the efficiency and reliability of building operation and maintenance.

[0006] To achieve the above purpose, the building operation and maintenance system for multi-modal data fusion and analysis based on BIM provided by the present invention is used in cooperation with a building and its BIM model, and includes

[0007] A multi-modal data acquisition layer, which is configured to be able to collect multi-modal data of the building in real time. The multi-modal data includes equipment operation data, environmental data, energy consumption data, and personnel activity data. The multi-modal data acquisition layer can also extract static information of the building from the BIM model.

[0008] A data fusion and processing layer, which is configured to be able to preprocess the multi-modal data and deeply fuse the preprocessed multi-modal data with static information to generate an operation and maintenance data set of the building.

[0009] The analysis and decision-making layer is configured to mine and analyze the operation and maintenance data set, form a fault prediction model, an energy consumption optimization model, and a space layout adjustment strategy for the building, and visualize them in the BIM model to generate an operation and maintenance decision-making plan for the building.

[0010] The operation and maintenance execution feedback layer is configured to convert the operation and maintenance decision-making plan into operation and maintenance tasks and assign them to corresponding operation and maintenance personnel.

[0011] Furthermore, sensing components and / or monitoring components for collecting the multimodal data are respectively provided on several areas and devices of the building.

[0012] Furthermore, the multimodal data acquisition layer performs semantic processing on the BIM model, assigns identifiers to each component and device in the BIM model, and adds spatial features and semantic information to generate the static information, which includes building structure information, equipment location information, pipeline layout information, equipment technical parameters, and maintenance history information.

[0013] Furthermore, the data fusion and processing layer identifies and removes the noise and outliers of the multimodal data through statistical analysis and rule judgment, and performs normalization processing on different multimodal data.

[0014] Furthermore, the data fusion and processing layer constructs a fusion network based on CNN-LSTM, extracts and fuses the spatial features of the static information and the local feature time series features of the multimodal data to form the operation and maintenance data set.

[0015] Furthermore, during the fusion process, the data fusion and processing layer synchronously adopts knowledge graph technology to associate and integrate the static information and the multimodal data.

[0016] Furthermore, the analysis and decision-making layer includes an equipment fault management module. The equipment fault management module uses the operation and maintenance data set, establishes a fault prediction model using machine learning algorithms, and predicts and determines the type and severity of equipment faults through learning historical fault data and real-time multimodal data.

[0017] Furthermore, the analysis and decision-making layer also includes an energy consumption optimization management module. The energy consumption optimization management module uses the operation and maintenance data set, establishes an energy consumption optimization model through reinforcement learning algorithms, and takes the energy consumption data, environmental data, and personnel activity data as inputs, and outputs the equipment operation data through the energy consumption optimization model.

[0018] Further, the analysis and decision-making layer further includes a space utilization and personnel flow optimization module. The space utilization and personnel flow optimization module combines the space information of the BIM model and personnel activity data, and uses GIS technology to analyze and judge the personnel activity rules and space usage conditions to generate a space layout adjustment strategy.

[0019] Further, the operation and maintenance execution feedback layer can also feedback the execution results of the operation and maintenance tasks by the operation and maintenance personnel to the data fusion processing layer and the analysis and decision-making layer.

[0020] The BIM-based multi-modal data fusion and analysis building operation and maintenance system provided by the present invention deeply fuses the static information of the building and real-time multi-modal data, can make full use of the space characteristics and semantic information of the BIM model, and more comprehensively and accurately reflect the operation state of the building to improve the operation and maintenance reliability. At the same time, through the analysis of the operation and maintenance data set, the building can be fault predicted, energy consumption optimized and space layout adjusted, and visualized in the BIM model to improve the efficiency of building operation and maintenance. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The present invention will be further described below in conjunction with the drawings and specific embodiments.

[0022] Figure 1 is the overall block diagram of the BIM-based multi-modal data fusion and analysis building operation and maintenance system provided by the present invention;

[0023] Figure 2 is the system block diagram of the multi-modal data acquisition layer in the present invention;

[0024] Figure 3 is the system block diagram of the data fusion processing layer in the present invention;

[0025] Figure 4 is the system block diagram of the analysis and decision-making layer in the present invention;

[0026] Figure 5 is the system block diagram of the operation and maintenance execution feedback layer in the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0027] In order to make the technical means, creative features, achieved purposes and effects of the present invention easy to understand, the present invention will be further described below in conjunction with specific illustrations.

[0028] See Figure 1 , which shows an example of the BIM-based multi-modal data fusion and analysis building operation and maintenance system provided by the present invention.

[0029] As can be seen from the figure, the building operation and maintenance system of this example, in cooperation with the building and its BIM model, mainly includes a multi-modal data acquisition layer 100, a data fusion and processing layer 200, an analysis and decision-making layer 300, and an operation and maintenance execution and feedback layer 400.

[0030] The multi-modal data acquisition layer 100 is configured to be able to collect multi-modal data of the building in real time. The multi-modal data includes equipment operation data, environmental data, energy consumption data, and personnel activity data. The multi-modal data acquisition layer 100 can also extract static information of the building from the BIM model.

[0031] Furthermore, the data fusion and processing layer 200 is configured to be able to preprocess the multi-modal data and deeply fuse the preprocessed multi-modal data with the static information to generate an operation and maintenance data set of the building.

[0032] In addition, the analysis and decision-making layer 300 is configured to be able to mine and analyze the operation and maintenance data set, form a fault prediction model, an energy consumption optimization model, and a space layout adjustment strategy of the building, and visualize them in the BIM model to generate an operation and maintenance decision-making plan for the building.

[0033] Furthermore, the operation and maintenance execution and feedback layer is configured to be able to convert the operation and maintenance decision-making plan into operation and maintenance tasks and assign them to corresponding operation and maintenance personnel.

[0034] Thus, the multi-modal data acquisition layer 100, the data fusion and processing layer 200, the analysis and decision-making layer 300, and the operation and maintenance execution and feedback layer 400 can cooperate with each other, make full use of the spatial characteristics and semantic information of the BIM model, more comprehensively and accurately reflect the operation state of the building, and perform fault prediction, energy consumption optimization, and space layout adjustment on the building to improve the efficiency and reliability of building operation and maintenance.

[0035] Combined Figure 2 , in order to collect multi-modal data, the multi-modal data acquisition layer 100 includes sensing components and / or monitoring components respectively set in several areas and equipment of the building to collect corresponding multi-modal data in real time.

[0036] As an example, current sensors, voltage sensors, and rotational speed sensors are respectively provided on the equipment of the building, so that the current sensors, voltage sensors, and rotational speed sensors respectively collect the current, voltage, and rotational speed of the equipment operation in real time, thereby obtaining equipment operation data.

[0037] Furthermore, temperature sensors, humidity sensors, smoke sensors, and / or gas concentration sensors are also distributed in the building, so that the temperature sensors, humidity sensors, smoke sensors, and / or gas concentration sensors can respectively collect the temperature, humidity, and air quality of the building in real time, thereby obtaining environmental data.

[0038] Correspondingly, the electricity meter, water meter, and gas meter installed in the building can respectively collect the electricity consumption, water consumption, and gas consumption of the building in real time, so as to obtain energy consumption data.

[0039] At the same time, an access control recording device is provided at the entrance of the building, and personnel monitoring devices are also distributed in various areas of the building, so that the access control recording device and the personnel monitoring device can respectively collect the personnel access situation and personnel positioning information in real time, so as to obtain personnel activity data.

[0040] Thus, the sensing components and / or monitoring components on several areas and devices cooperate with each other to collect the equipment operation data, environmental data, energy consumption data, and personnel activity data of the building in real time. In order to improve the data accuracy, the multi-modal data acquisition layer 100 will set the acquisition frequency and accuracy of the sensing components and / or monitoring components according to the characteristics of the data collected by different sensing components and / or monitoring components, so as to improve the acquisition efficiency and accuracy, thereby obtaining the multi-modal data of the building in real time and transmitting it to the data fusion and processing layer 200 in real time.

[0041] As an example, for data such as rotational speed in the equipment operation data that changes relatively fast, the multi-modal data acquisition layer 100 correspondingly increases the acquisition frequency and acquisition accuracy of the rotational speed sensor, so that the rotational speed sensor can collect rotational speed data multiple times per second, while ensuring that the rotational speed data has a high accuracy, in order to obtain more rotational speed data and avoid data loss, thereby improving the accuracy of data acquisition.

[0042] Correspondingly, for data such as temperature in the environmental data that changes relatively slowly, the multi-modal data acquisition layer 100 correspondingly controls the acquisition frequency and acquisition accuracy of the temperature sensor to be kept at a moderate level. For example, the acquisition frequency of the temperature sensor is set to be once per minute, which can not only ensure the accurate acquisition of temperature data, but also save the data storage and processing resources of the multi-modal data acquisition layer 100.

[0043] For data such as electricity consumption in the energy consumption data whose change speed is unstable, the multi-modal data acquisition layer 100 periodically controls the acquisition frequency of the electricity meter according to the change speed of the electricity consumption. For example, the multi-modal data acquisition layer 100 controls the electricity meter to collect data once per minute during the peak electricity consumption period, and once every 5 - 10 minutes during the low valley period, so that the data acquisition frequency is adapted to the data change speed and the data acquisition accuracy is improved.

[0044] Furthermore, the multi-modal data acquisition layer 100 can also extract the static information of the building from the BIM model, so that the multi-modal data is fused and associated with the static information.

[0045] Combined Figure 2,Specifically, the multimodal data acquisition layer 100 performs semantic processing on the ,BIM model to determine the clear meaning of each component and equipment in the ,BIM model. Firstly, a structured semantic framework is built ,according to the general specifications of the construction industry, such as design ,construction standards and the actual needs of building operation and ,maintenance, such as equipment management and maintenance.

[0046] In this example, the multimodal data acquisition layer 100 adopts the internationally accepted IFC (Industrial Foundation Classes) standard as the framework basis, uniformly defining the attributes, relationships and classification rules of components and equipment in the BIM model, thereby providing standardized support for subsequent semantic analysis, data association and other processing.

[0047] Furthermore, the multimodal data acquisition layer 100 assigns a unique identifier to each component and equipment in the BIM model according to the IFC standard, and marks the function, type, material and other attributes corresponding to each component and equipment. For example, the multimodal data acquisition layer 100 marks the fire sprinkler in the BIM model with attributes such as "fire sprinkler" type and "working pressure".

[0048] Next, the multimodal data collection layer 100 adds spatial features and semantic information to each component and device.

[0049] Specifically, when adding spatial features to each component and equipment, the multimodal data acquisition layer 100 uses the three-dimensional geometric information of the BIM model to extract the position, size, shape and other features of the components and equipment, and uses the BIM model coordinates to determine the specific positions of the components and equipment.

[0050] Furthermore, the multimodal data collection layer 100 adds semantic information to each component and device, and supplements the operation and maintenance related information such as the maintenance cycle and service life of each component and device.

[0051] Therefore, the multimodal data acquisition layer 100 can obtain the identifier, spatial characteristics and semantic information of each component and equipment in the BIM model through semantic processing of the BIM model, and can effectively obtain the static information of the building.

[0052] Furthermore, the multimodal data acquisition layer 100 associates the multimodal data collected in real time with each component and equipment in the BIM model, for example, associating the smoke sensor data with nearby fire-fighting equipment, to facilitate the subsequent fusion processing of the multimodal data and static information.

[0053] The multimodal data acquisition layer 100 thus constructed can perform semantic processing on the BIM model to accurately obtain the static information of the building, including building structure information, equipment location information, pipeline layout information, equipment technical parameters and maintenance history information, and transmit it to the data fusion processing layer 200 in real time.

[0054] In this example, the multimodal data acquisition layer 100 and the data fusion processing layer 200 are communicatively connected via a wired or wireless communication network to ensure that the multimodal data acquisition layer 100 can transmit multimodal data and static information to the data fusion processing layer 200 in real time.

[0055] Combined with Figure 3 , correspondingly, the data fusion processing layer 200 is configured to preprocess and fuse multimodal data and static information, so as to utilize the spatial features and semantic information of the BIM model, combined with real-time multimodal data, to more comprehensively and accurately reflect the operating state of the building, facilitating subsequent operation and maintenance management of the building.

[0056] Specifically, the data fusion processing layer 200 preprocesses the multimodal data. In the preprocessing stage, the data fusion processing layer 200 first performs cleaning processing, adopting a dual mechanism of statistical analysis and rule judgment to identify and remove noise data, outliers, and duplicate records in the multimodal data, so as to improve the accuracy of the multimodal data.

[0057] As an example, the data fusion processing layer 200 statistically analyzes the mean, standard deviation, etc. of the multimodal data to set a reasonable range for the multimodal data, and regards the multimodal data outside the reasonable range as outliers for removal. At the same time, the data fusion processing layer 200 uses rule judgment to remove noise data. For example, according to the operating logic of the device, it checks whether the device operation data conforms to the normal operation rules, and regards the device operation data that does not conform to the normal operation rules as noise data and removes it.

[0058] Furthermore, the data fusion processing layer 200 performs normalization processing on the multimodal data, performs format conversion and standardization processing on different multimodal data to unify the data coding method and data structure, and eliminate the dimensional difference between different modal data.

[0059] As an example, for numerical data, such as temperature, humidity, etc., the data fusion processing layer 200 adopts the Min-Max normalization method to map the multimodal data to the interval [0,1]. For categorical data, such as air quality, personnel positioning information, etc., the data fusion processing layer 200 performs one-hot encoding to convert the data category into a vector form to unify the data coding method and data structure.

[0060] Next, the data fusion processing layer 200 stores the multimodal data after cleaning and normalization processing in a unified format for subsequent fusion processing.

[0061] Furthermore, the data fusion processing layer 200 deeply fuses the preprocessed multi-modal data with the static information. First, feature extraction is performed. The data fusion processing layer 200 extracts the spatial features of the static information (such as the location information of the device) and the local features of the real-time multi-modal data through the CNN network. At the same time, the LSTM network is used to capture the time series features of the real-time multi-modal data.

[0062] As an example, the local features of the multi-modal data include the specific distribution positions of the sensing components and / or monitoring components to reflect the spatial distribution characteristics of the corresponding multi-modal data. For example, the distribution positions of temperature sensors in the building area are extracted to reflect the spatial distribution characteristics of temperature data in the building.

[0063] Correspondingly, the time series features of the multi-modal data include the change situations of the multi-modal data to reflect the change trends of the corresponding multi-modal data over time. For example, extracting the change situation of the rotation speed of a device over a period of time can reflect the change trend of the operating rotation speed of the device over time.

[0064] In this way, the data fusion processing layer 200 extracting the local features and time series features of the multi-modal data can clearly understand the spatio-temporal characteristics of the building operation. At the same time, the occurrence area of equipment failures can be judged through the local features, and the development trend of equipment failures can be analyzed through the time series features, providing accurate diagnostic support for the fault prediction of the analysis and decision-making layer 300.

[0065] Combined Figure 3 , then, the data fusion processing layer 200 constructs a CNN-LSTM based fusion network, dynamically assigns weight coefficients to different features through the attention mechanism, and realizes the feature association and deep fusion of the static information and the multi-modal data to form an operation and maintenance data set of the building.

[0066] Specifically, the data fusion processing layer 200 uses the attention mechanism to compare the contributions of different features to target tasks such as equipment fault prediction and energy consumption optimization, and assigns corresponding weights, thereby highlighting the information important for the current task and improving the accuracy of the operation and maintenance data set after fusion.

[0067] As an example, the data fusion processing layer 200 calculates the correlation between different features and the target task, and sets high weights for the features with higher correlation. For example, if the target task is fault prediction, the data fusion processing layer 200 calculates that the time series features of the device operation data have a greater impact on the judgment of equipment faults, and sets the weight system of the time series features of the device operation data to 0.7. Correspondingly, the weight coefficient of the spatial features of the corresponding device in the static information is set to 0.3, thereby highlighting the important data to improve the operation and maintenance reliability.

[0068] Furthermore, the data fusion processing layer 200 synchronously utilizes knowledge graph technology to associate and integrate static information and multimodal data, further mining the semantic relationships and potential knowledge between static information and multimodal data.

[0069] Specifically, the data fusion processing layer 200 first constructs a knowledge graph framework, entityifies static information (such as device technical parameters) and multimodal data (such as real-time device operation data), and establishes nodes. Then, edges are created based on data association relationships (such as the association between a device and its corresponding device operation data). For example, a device node is connected to its corresponding device operation data node. Next, graph embedding technology is used to transform the nodes and edges into vector representations to facilitate the mining and processing by the data fusion processing layer 200.

[0070] Furthermore, when the data fusion processing layer 200 performs mining and processing, potential knowledge is discovered through the associated path analysis of the graph. For example, the root cause of a fault is mined from the association between device fault history data and current device operation data, thereby more comprehensively understanding the relationship between static information and multimodal data and improving the accuracy of analysis and decision-making.

[0071] The data fusion processing layer 200 thus constituted associates and fuses multimodal data and static information to form an operation and maintenance data set for the building, and the operation and maintenance data set is transmitted to the analysis and decision-making layer 300 in real time, facilitating the analysis and decision-making layer 300 to analyze the operation and maintenance data set, so as to adjust the operation state of the building.

[0072] Combined Figure 4 , specifically, the analysis and decision-making layer 300 includes a fault management module 310, an energy consumption optimization management module 320, and a space utilization and personnel flow optimization module 330. The fault management module 310, the energy consumption optimization management module 320, and the space utilization and personnel flow optimization module 330 can cooperate with each other and operate synchronously to regulate the device faults, energy consumption, and space layout of the building.

[0073] Among them, the fault management module 310 is configured to be able to adopt the operation and maintenance data set, establish a fault prediction model for the building using machine learning algorithms, and predict and determine the type and severity of device faults through the learning of historical fault data and real-time multimodal data.

[0074] When establishing the existing device fault prediction model, a large amount of labeled data is required for training, and its generalization ability is poor in different types of devices or new application scenarios. Therefore, the fault management module 310 of this operation and maintenance system adopts a method combining transfer learning and ensemble learning, utilizes the existing knowledge and models in related fields, quickly adapts to new device fault prediction tasks, reduces the demand for training data, and improves the accuracy and generalization ability of the fault prediction model.

[0075] Specifically, the fault management module 310 selects a pre-trained model that has been trained in the field of equipment fault diagnosis, such as a convolutional neural network model like ResNet or VGG, freezes some parameters of the pre-trained model, and retains its general feature extraction ability.

[0076] Next, the fault management module 310 uses the transfer learning method to fine-tune the pre-trained model based on the historical fault data of the equipment on the basis of the pre-trained model, so that the pre-trained model can learn the fault characteristics unique to the building's equipment.

[0077] Furthermore, the fault management module 310 adopts the ensemble learning method to combine multiple fine-tuned pre-trained models, and obtains the final fault prediction model by voting or weighted average.

[0078] During the training and combination process, the fault management module 310 continuously optimizes the parameters of the pre-trained model to improve the accuracy of the final fault prediction model.

[0079] Furthermore, the fault management module 310 combines the multi-modal data collected in real time and the historical fault data of the equipment, and can predict the type and severity of the equipment fault through the fault prediction model.

[0080] At the same time, the fault management module 310 analyzes the fault data obtained by the fault prediction model through deep learning algorithms of convolutional neural network and recurrent neural network to construct a fault diagnosis model. When the equipment fails, the fault management module 310 quickly and accurately judges the cause and location of the fault according to the collected fault feature data.

[0081] To improve the maintenance efficiency, the fault management module 310 applies the fault prediction model to the BIM model and visualizes it in the BIM model to intuitively display the location and relevant information of the faulty equipment, facilitating the operation and maintenance personnel to handle it.

[0082] Furthermore, the energy consumption optimization management module 320 uses the operation and maintenance data set to establish an energy consumption optimization model through the reinforcement learning algorithm, and takes the energy consumption data, environmental data and personnel activity data as inputs, and outputs the equipment operation data through the energy consumption optimization model to adjust the energy consumption of the equipment and improve the operation and maintenance reliability.

[0083] Specifically, in this example, the energy consumption optimization management module 320 combines the energy consumption data, environmental data and personnel activity data of the multi-modal data, and adopts the deep deterministic policy gradient algorithm to construct the energy consumption optimization model of the building, so that the energy consumption optimization model can interact with the real-time acquired environmental data and output the equipment operation data at this time, so as to achieve the purpose of adjusting the equipment operation state.

[0084] For example, the energy consumption optimization management module 320 first determines the input of the energy consumption optimization model, preprocesses and performs feature engineering on the energy consumption data, environmental data, and personnel activity data to convert them into a format acceptable to the energy consumption optimization model. Then, it selects the Deep Deterministic Policy Gradient algorithm, constructs a network structure, such as setting the number of neurons in the input layer, hidden layer, and output layer of the energy consumption optimization model, and then trains the energy consumption optimization model.

[0085] During the training phase, the energy consumption optimization management module 320 interacts the energy consumption optimization model with the real-time collected environmental data, and continuously adjusts the network parameters of the energy consumption optimization model according to feedback reward signals such as energy consumption savings and comfort evaluation. Through multiple iterative trainings, the device operation data output by the energy consumption optimization model can achieve energy consumption optimization, thereby establishing the energy consumption optimization model.

[0086] At the same time, the energy consumption optimization management module 320 applies the energy consumption optimization model to the BIM model, makes the energy consumption data visible in the BIM model, and combines the spatial information of the BIM model to analyze the energy consumption distribution in different regions, which is convenient for adjusting the device operation state and provides a reference for the energy-saving renovation of the building.

[0087] Furthermore, the energy consumption optimization management module 320 introduces an adaptive mechanism to the energy consumption optimization model, enabling the energy consumption optimization model to automatically adjust the learning rate and exploration strategy according to the change frequency and amplitude of the building operation state, and improving the adaptability and stability of the energy consumption optimization model.

[0088] Compared with existing energy consumption management systems that often adopt fixed control strategies, the energy consumption optimization management module 320 thus constituted can adjust the operation parameters of the device in real time according to the real-time collected multi-modal data, achieving efficient utilization of energy.

[0089] Furthermore, the analysis and decision-making layer 300 also includes a space utilization and personnel flow optimization module 330. The space utilization and personnel flow optimization module 330 combines the spatial information of the BIM model and personnel activity data, and uses GIS technology to analyze and judge the personnel activity rules and space usage conditions to generate a space layout adjustment strategy.

[0090] Specifically, the space utilization and personnel flow optimization module 330 uses GIS technology to analyze the personnel activity data and the space usage conditions of the BIM model, can discover unreasonable places in space utilization, and thus propose suggestions for space layout adjustment to improve space utilization rate. At the same time, the space utilization and personnel flow optimization module 330 can optimize the traffic flow line and safety evacuation passage of the building according to the personnel flow rules, improving the personnel passage efficiency and safety.

[0091] For example, the space utilization and personnel flow optimization module 330 first imports the space information of the BIM model into the GIS platform, integrates it with the personnel activity data, and through the spatial analysis function of GIS, such as hot spot analysis, locates the areas with intensive personnel activities, and then through buffer analysis, checks the service scope of space facilities.

[0092] Furthermore, if a certain area is too crowded with people and the utilization of space facilities is insufficient, the space utilization and personnel flow optimization module 330 will propose suggestions for merging or re-dividing the space; correspondingly, if the utilization rate of the safety evacuation passage is low, the space utilization and personnel flow optimization module 330 will propose suggestions for optimizing the passage layout and form a space layout adjustment strategy for the reference of the operation and maintenance personnel.

[0093] The analysis and decision-making layer 300 thus constituted, through the cooperation of the fault management module 310, the energy consumption optimization management module 320, and the space utilization and personnel flow optimization module 330, performs real-time regulation on the equipment faults, energy consumption, and space layout of the building to improve the operation and maintenance efficiency and reliability.

[0094] Furthermore, the analysis and decision-making layer 300 can also generate a scientific and reasonable operation and maintenance decision-making plan, such as an equipment maintenance plan, an energy consumption regulation strategy, and a space layout adjustment suggestion, based on the fault prediction model, the energy consumption optimization model, and the space layout adjustment strategy respectively obtained by the fault management module 310, the energy consumption optimization management module 320, and the space utilization and personnel flow optimization module 330.

[0095] Combined with Figure 4 , for example, the analysis and decision-making layer 300 summarizes the analysis results of each module, such as the fault prediction information of the fault management module 310, the energy-saving strategy of the energy consumption optimization management module 320, and the layout adjustment suggestion of the space utilization and personnel flow optimization module 330; the analysis and decision-making layer 300 integrates and prioritizes these results. For example, it is set to prioritize the handling of serious equipment faults, and based on the sorting results, combined with operation and maintenance resources, such as the situation of operation and maintenance personnel and operation and maintenance materials, a specific operation and maintenance decision-making plan is formulated. For example, the operation and maintenance personnel first repair the high-priority faulty equipment and then adjust the energy consumption equipment.

[0096] Furthermore, the analysis and decision-making layer 300 stores these operation and maintenance decision-making plans in the form of tables or documents and transmits them to the operation and maintenance execution and feedback layer 400 to guide the development of operation and maintenance work.

[0097] Combined with Figure 5, to respond to the operation and maintenance decision-making plan of the analysis and decision-making layer 300 and perform operation and maintenance on the building, this operation and maintenance system further includes an operation and maintenance execution feedback layer 400. The analysis and decision-making layer 300 transmits the operation and maintenance decision-making plan to the operation and maintenance execution feedback layer 400 in real time, enabling the operation and maintenance execution feedback layer 400 to convert the operation and maintenance decision-making plan into operation and maintenance tasks and allocate them to corresponding operation and maintenance personnel for execution, thereby improving operation and maintenance efficiency.

[0098] Specifically, the operation and maintenance execution feedback layer 400 has functions of task scheduling and resource allocation. It can reasonably allocate the operation and maintenance decision-making plan into corresponding different operation and maintenance personnel and operation and maintenance tasks according to factors such as the skill level, workload, and geographical location of the operation and maintenance personnel, improving operation and maintenance efficiency.

[0099] Furthermore, the operation and maintenance personnel receive the corresponding operation and maintenance tasks through the mobile application and operate according to the requirements of the operation and maintenance tasks. During the operation and maintenance process, the operation and maintenance personnel can upload the on-site situation and processing results in real time, including repair records, information on replaced parts, energy consumption adjustment effects, etc., enabling the operation and maintenance execution feedback layer 400 to monitor and evaluate the operation and maintenance execution situation in real time and feedback the execution results to the data fusion processing layer 200 and the analysis and decision-making layer 300 as historical fault information and historical operation and maintenance information for the update and optimization of each model in the data fusion processing layer 200 and the analysis and decision-making layer 300, forming a closed-loop operation and maintenance management system.

[0100] Combined Figure 5 , as an example, the operation and maintenance execution feedback layer 400 first analyzes the operation and maintenance decision-making plan, extracts key information such as task type (such as equipment repair, energy consumption adjustment), target object (specific equipment or area), etc., and then uses the task scheduling algorithm for task allocation according to factors such as the skill level, workload, and geographical location of the operation and maintenance personnel. For example, allocate the equipment repair task to a person who is familiar with the equipment repair skill, has a relatively low workload, and is relatively close.

[0101] Furthermore, the operation and maintenance execution feedback layer 400 converts the specific operation and maintenance task information into a format that can be received by the mobile application, such as the JSON data format, and pushes it to the mobile end of the corresponding operation and maintenance personnel through the network. Correspondingly, after the operation and maintenance personnel complete the corresponding operation and maintenance tasks, the mobile end collects the feedback data and then converts it into a format that can be processed by the operation and maintenance execution feedback layer 400 and transmits it back to the data fusion processing layer 200 and the analysis and decision-making layer 300.

[0102] The operation and maintenance execution feedback layer 400 thus constituted can convert the operation and maintenance decision-making plan into reasonable operation and maintenance tasks, allocate them to corresponding operation and maintenance personnel for execution, and at the same time feedback the execution results, thereby improving operation and maintenance efficiency and effect.

[0103] The building operation and maintenance system based on BIM for multi-modal data fusion and analysis provided by the present invention deeply fuses the static information of the building and real-time multi-modal data, can make full use of the spatial characteristics and semantic information of the BIM model, and more comprehensively and accurately reflect the operation state of the building to improve the reliability of operation and maintenance. At the same time, through the analysis of the operation and maintenance data set, the building can be predicted for faults, optimized for energy consumption, and adjusted for spatial layout, and visualized in the BIM model to improve the efficiency of building operation and maintenance.

[0104] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited by the above embodiments, and what is described in the above embodiments and the specification only illustrates the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention will have various changes and improvements, and these changes and improvements all fall within the scope of the present invention claimed. The scope of the present invention claimed is defined by the appended claims and their equivalents.

Claims

1. The BIM-based multimodal data fusion and analysis building operation and maintenance system is used to cooperate with the BIM model of buildings and buildings, and is characterized by: include A multimodal data acquisition layer, wherein the multimodal data acquisition layer is configured to acquire multimodal data of the building in real time, wherein the multimodal data includes equipment operation data, environmental data, energy consumption data, and personnel activity data, and the multimodal data acquisition layer can also extract static information of the building from the BIM model. A data fusion processing layer, wherein the data fusion processing layer is configured to pre-process the multimodal data and deeply fuse the pre-processed multimodal data with static information to generate an operation and maintenance data set of the building. An analysis and decision-making layer, wherein the analysis and decision-making layer is configured to mine and analyze the operation and maintenance data set to form a fault prediction model, an energy consumption optimization model, and a space layout adjustment strategy for the building, and visualize them in the BIM model to generate an operation and maintenance decision-making plan for the building, The operation and maintenance execution feedback layer is configured to convert the operation and maintenance decision plan into an operation and maintenance task and assign it to corresponding operation and maintenance personnel.

2. The BIM-based multimodal data fusion and analysis building operation and maintenance system according to claim 1, characterized in that: Several areas and equipment of the building are respectively provided with sensing components and / or monitoring components for collecting the multimodal data.

3. The BIM-based multimodal data fusion and analysis building operation and maintenance system according to claim 2 is characterized in that: The multimodal data acquisition layer performs semantic processing on the BIM model, assigns an identifier to each component and equipment in the BIM model, and adds spatial features and semantic information to generate the static information, which includes building structure information, equipment location information, pipeline layout information, equipment technical parameters and maintenance history information.

4. The multimodal data fusion and analysis building operation and maintenance system based on BIM according to claim 1, characterized in that: The data fusion processing layer identifies and removes noise and outliers of the multimodal data through statistical analysis and rule judgment, and normalizes different multimodal data.

5. The multimodal data fusion and analysis building operation and maintenance system based on BIM according to claim 4, characterized in that: The data fusion processing layer constructs a fusion network based on CNN-LSTM, extracts and fuses the spatial features of the static information with the local features and temporal features of the multimodal data to form the operation and maintenance data set.

6. The BIM-based multimodal data fusion and analysis building operation and maintenance system according to claim 5, characterized in that: During the fusion process, the data fusion processing layer simultaneously uses knowledge graph technology to associate and integrate static information and multimodal data.

7. The multimodal data fusion and analysis building operation and maintenance system based on BIM according to claim 1, characterized in that: The analysis and decision-making layer includes an equipment fault management module, which adopts the operation and maintenance data set and uses a machine learning algorithm to establish a fault prediction model, and predicts and determines the type and severity of equipment failures by learning historical fault data and real-time multimodal data.

8. The multimodal data fusion and analysis building operation and maintenance system based on BIM according to claim 7, characterized in that: The analysis and decision-making layer also includes an energy consumption optimization management module, which adopts the operation and maintenance data set to establish an energy consumption optimization model through a reinforcement learning algorithm, and uses the energy consumption data, environmental data and personnel activity data as input to output the equipment operation data through the energy consumption optimization model.

9. The multimodal data fusion and analysis building operation and maintenance system based on BIM according to claim 8, characterized in that: The analysis and decision-making layer also includes a space utilization and personnel flow optimization module, which combines the spatial information and personnel activity data of the BIM model and uses GIS technology to analyze and judge the patterns of personnel activity and space usage to generate a space layout adjustment strategy.

10. The multimodal data fusion and analysis building operation and maintenance system based on BIM according to claim 1, characterized in that: The operation and maintenance execution feedback layer can also feed back the execution results of the operation and maintenance tasks by the operation and maintenance personnel to the data fusion processing layer and the analysis and decision-making layer.

Images