BIM data efficient storage and access method based on distributed architecture

Abstract

The invention discloses a BIM data efficient storage and access method based on a distributed architecture, and the method comprises the following steps: S1, data distributed storage design: employing a highly extensible and fault-tolerant distributed file system as a basis, and distributing BIM data to a plurality of storage nodes; the system resource consumption is reduced; the user experience is improved; s3, establishing a multi-level and multi-dimensional data index system so as to quickly and accurately retrieve the BIM data; and S4, optimizing data synchronization and consistency to meet the requirements of high real-time performance and high reliability of the BIM data. Through the design of the distributed architecture, the BIM data are stored on a plurality of nodes in a scattered manner, and the storage capacity and the processing capacity of the data are remarkably improved. Through data fragmentation and load balancing technologies, the problems are effectively relieved, so that the system can efficiently process massive BIM data.

Description

Technical Field

[0001] The present invention relates to the field of information technology, and in particular to a method for efficiently storing and accessing BIM data based on a distributed architecture. Background Art

[0002] With the widespread application of building information modeling (BIM) technology around the world, especially in large and complex construction projects, the management and access of BIM data has become a key issue that needs to be solved urgently. At present, the traditional centralized storage architecture seems to be unable to cope with large-scale BIM data, and its problems such as poor scalability, slow data access speed, and complex management are becoming increasingly prominent. Specifically, traditional BIM data storage methods often rely on a single or a few high-performance servers. When faced with massive data, these servers not only have high storage costs, but also have difficulty in ensuring instant access and high availability of data. In addition, as the project progresses, the continuous growth of data volume has put forward higher requirements for the scalability of the system, and traditional architectures often find it difficult to meet this demand. They generally face the following problems and shortcomings when processing large-scale BIM data: 1) Limited storage scalability: When faced with large-scale BIM data, the scalability of traditional centralized storage systems becomes a bottleneck. As the amount of data continues to increase, the hardware equipment needs to be continuously expanded, which is not only costly, but also complicated to expand, making it difficult to meet the needs of rapid project development.

[0003] 2) Low data access efficiency: In a large-scale data environment, the read and write performance of traditional storage systems is difficult to support high-frequency data access needs. Especially in multi-user collaborative work scenarios, data access delays will seriously affect project management efficiency and real-time decision-making capabilities.

[0004] 3) Insufficient data reliability and availability: Single-point storage systems have the risk of single-point failure. Once a failure occurs, the data will be inaccessible, seriously affecting the continuity of the project and the security of the data. In addition, the lack of effective redundancy mechanisms and data backup solutions further reduces the reliability and availability of data.

[0005] 4) Insufficient indexing and retrieval capabilities: Existing data indexing and retrieval technologies often have difficulty providing fast and accurate query services when facing large-scale BIM data. The construction and maintenance costs of indexes are high, and it is difficult to dynamically adjust according to the access mode of data, resulting in low data retrieval efficiency.

[0006] 5) Difficulty in data synchronization and consistency management: In a multi-storage node environment, data synchronization and consistency management is a complex issue. Existing data synchronization mechanisms often have problems such as high synchronization delay and low synchronization efficiency, and it is difficult to ensure the consistency of data in a distributed system.

[0007] In summary, the existing BIM data storage and access technologies have problems such as limited scalability, low access efficiency, insufficient reliability and availability, insufficient indexing and retrieval capabilities, and difficulties in data synchronization and consistency management when processing large-scale data. These problems seriously restrict the efficient implementation of BIM project management and real-time decision-making capabilities.

[0008] Therefore, the present invention proposes an efficient storage and access method for BIM data based on a distributed architecture, aiming to solve the problems and shortcomings existing in the prior art and improve data reading and writing performance and project management efficiency through distributed storage and access optimization technology. Summary of the invention

[0009] The purpose of the present invention is to provide a method for efficiently storing and accessing BIM data based on a distributed architecture to solve the technical problems existing in the prior art.

[0010] In order to achieve the above object, the present invention adopts the following technical solutions: The present invention provides a method for efficiently storing and accessing BIM data based on a distributed architecture, comprising the following steps: S1. Data distributed storage design: A highly scalable and fault-tolerant distributed file system is used as the basis to distribute BIM data to multiple storage nodes: S2. Read path optimization: to increase data access speed, reduce system resource consumption and improve user experience; S3. Establish a multi-level and multi-dimensional data index system to quickly and accurately retrieve BIM data; S4. Optimize data synchronization and consistency to meet the high real-time and high reliability requirements of BIM data.

[0011] Furthermore, the distributed file system adopts Hadoop Distributed File System or open source Ceph distributed storage system.

[0012] Furthermore, the data distributed storage design includes node selection and configuration optimization, data sharding and load balancing, and data encryption and privacy protection; The node selection and configuration optimization adopts an intelligent node selection and configuration strategy; firstly, the hardware resources and network bandwidth of each node are evaluated, and then according to the access characteristics and data volume of BIM data, storage nodes are dynamically selected for data storage; The data sharding and load balancing introduces data sharding technology, which splits large BIM files into multiple small fragments according to preset rules and distributes and stores them on different nodes; The data encryption and privacy protection uses an encryption algorithm to encrypt BIM data, providing an extremely high security level for BIM data, allowing users to store and access their sensitive data with confidence.

[0013] Furthermore, the data encryption and privacy protection are specifically implemented as follows: Step 1: Encrypt BIM data before data storage; the encryption process is performed before data fragmentation, or each data fragment is encrypted separately, depending on system design and performance considerations; Step 2: The encrypted data will be stored on distributed nodes to ensure security during storage and transmission, effectively preventing data leakage and illegal access; Step 3: Fine-grained access control policies ensure that project team members can only access data that matches their permissions, further enhancing data privacy protection.

[0014] Furthermore, the read path optimization includes the innovation of intelligent cache strategy, innovation of concurrent read and write technology and read and write conflict resolution mechanism; The innovative implementation process of the intelligent caching strategy is as follows: Step 1: Data Collection and Analysis Collect log information of users accessing BIM data, including access time, access frequency, and access mode; Use machine learning algorithms to conduct in-depth analysis of user access patterns and identify frequently accessed data patterns and access trends; Step 2: Intelligent prediction and caching Based on the analysis results, predict which BIM data will be frequently accessed; Cache the data predicted to be frequently accessed in the local cache in advance to reduce the number of accesses to remote storage and reduce access latency; Step 3: Dynamic Adjustment and Optimization Real-time monitoring of cache usage, including cache hit rate and cache data validity; Dynamically adjust cache policies based on monitoring results, such as adjusting cache size and replacement algorithms, to ensure that the data in the cache is always the most valuable; The implementation process of the innovation of the concurrent reading and writing technology is as follows: Step 1: Multithreading Using multithreading technology, multiple worker threads are created to handle concurrent read and write requests; Each worker thread independently processes a read and write request, achieving parallel processing of requests and improving CPU utilization; Step 2: Asynchronous IO operation Using asynchronous IO technology, read and write operations are allowed to be performed in the background without blocking the execution of the main thread; The main thread continues to process other tasks or requests, improving the system's responsiveness and throughput; Step 3: Resource coordination and management Coordinate and manage resources for multi-threaded and asynchronous IO operations to ensure reasonable allocation and efficient use of resources; reduce system overhead and improve concurrent processing capabilities through thread pool and IO reuse technologies; The read-write conflict resolution mechanism is specifically as follows: Step 1: Concurrency Control Strategy Introduce optimistic locking or pessimistic locking concurrency control strategies to coordinate and manage concurrent read and write operations; Optimistic locking is suitable for scenarios with more reads than writes. It uses version numbers or timestamps to detect whether data has been modified during reading and writing. Pessimistic locking is suitable for scenarios with a lot of writes. It uses a locking mechanism to ensure that data is not interfered with by other operations when being written. Step 2: Data consistency and integrity assurance Ensure the consistency and integrity of data in a concurrent environment; ensure the correctness and reliability of data through transaction management and logging; Step 3: Exception handling and recovery mechanism Supports transaction rollback and recovery mechanisms; when an abnormal situation occurs, the transaction can be rolled back in time and the data can be restored to the correct state.

[0015] Furthermore, the establishment of a multi-level, multi-dimensional data index system includes the combination of static index and dynamic index, the fusion of spatial index and time index, and the maintenance and optimization of index; wherein, The specific implementation steps of combining the static index with the dynamic index are as follows: Step 1: During the data storage phase, a static index entry is created for each data segment and stored in the index database; Step 2: These static index entries support basic query operations; Step 3: Dynamic indexing dynamically adjusts the index structure based on user access patterns and data usage; Step 4: By monitoring the user's query requests and data access frequency, the frequently accessed data segments are automatically identified and promoted to the top level of the index tree or a position with a higher priority; The fusion of the spatial index and the time index is as follows: the spatial index supports queries based on spatial locations by dividing the spatial regions and establishing index entries for each region; the time index builds an index based on the timestamp information of the BIM data to support time series analysis; The maintenance and optimization of the index specifically includes: when the BIM data changes, the index information is synchronously updated to ensure the correctness of the query results; at the same time, the performance and efficiency of the index are regularly evaluated and the index is optimized and adjusted according to the evaluation results.

[0016] Furthermore, the optimization of data synchronization and consistency includes the combination of incremental synchronization and full synchronization, log-based replication mechanism, conflict detection and resolution mechanism, real-time optimization, and selection and implementation of consistency model; wherein, The efficiency of data synchronization is improved by combining the incremental synchronization with the full synchronization; The log-based replication mechanism specifically means: on each storage node, an operation log is maintained to record all modification operations on BIM data; when data synchronization is required, the change records in the operation log are simply transferred to the target node, and these change records are replayed on the target node to achieve data synchronization; The conflict detection and resolution mechanism specifically refers to: when multiple nodes modify the same BIM data at the same time, potential conflicts are detected by comparing the change records in the operation log; once a conflict is found, the conflict is automatically resolved according to the preset conflict resolution strategy to ensure the consistency and accuracy of the data; The real-time optimization specifically refers to: shortening the synchronization delay by optimizing the network communication protocol, reducing the data verification and error processing time during the synchronization process, and improving the concurrent performance of log processing; The selection and implementation of the consistency model specifically refers to: ensuring the consistency and atomicity of cross-node operations by introducing distributed locks and transaction coordinator mechanisms; reducing the risk of data inconsistency by optimizing data replication and synchronization strategies; and ensuring data consistency in the long-term operation of the system by regularly performing consistency checks and repair operations.

[0017] By adopting the above technical solution, the present invention has the following beneficial effects: The present invention stores BIM data in multiple nodes through the design of distributed architecture, which significantly improves the storage and processing capabilities of data. Traditional centralized storage solutions often have read and write performance bottlenecks and single point failure risks when facing large-scale BIM data. The present invention effectively alleviates these problems through data sharding and load balancing technology, allowing the system to efficiently process massive BIM data. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1 A flowchart of the method for efficient storage and access of BIM data based on a distributed architecture provided by the present invention.

[0020] Figure 2 A flow chart of the data distributed storage design provided by the present invention; Figure 3 A flow chart of read-write path optimization provided by the present invention. DETAILED DESCRIPTION

[0021] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0022] The specific implementation of the present invention is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described here is only used to illustrate and explain the present invention, and is not used to limit the present invention.

[0023] Combination Figure 1-3As shown, this embodiment provides a method for efficient storage and access of BIM data based on a distributed architecture. The core of the present invention is to achieve efficient storage and access of BIM data through a distributed architecture. Specifically, we use advanced distributed file systems, such as Hadoop HDFS or Ceph (Note: Hadoop HDFS (Hadoop Distributed File System) is part of the Hadoop project and is designed for storing very large data sets. Its main features include high fault tolerance, streaming data access, suitability for large data sets, and a simple file model; Ceph is a high-performance distributed storage system that provides a unified storage solution and supports block storage, file storage, and object storage. Its main features include high scalability, high performance, a unified storage interface, and strong consistency) to distribute BIM data to multiple storage nodes. This design not only improves the scalability of data storage, but also enhances the reliability and availability of data through a redundant storage mechanism. Each storage node has independent processing capabilities and can achieve localized access to data to a certain extent, thereby reducing data transmission delays. Specifically includes the following steps: 1. Data distributed storage design In the design of data distributed storage, we use highly scalable and fault-tolerant distributed file systems as the basis, such as Hadoop HDFS (Hadoop Distributed File System) or the open source Ceph distributed storage system. These systems not only support the storage of massive data, but also have high reliability and high availability.

[0024] 1.1 Node selection and configuration optimization In order to ensure high performance and balanced load of storage nodes, the present invention adopts intelligent node selection and configuration strategy. The system first evaluates the hardware resources (such as CPU, memory, disk I / O, etc.) and network bandwidth of each node, and then dynamically selects the most suitable storage node for data storage based on the access characteristics and data volume of BIM data. In addition, the system also supports automatic expansion and contraction of nodes to cope with dynamic changes in data volume.

[0025] 1.2 Data Sharding and Load Balancing In order to further improve data access efficiency, the present invention introduces data sharding technology. Large BIM files are split into multiple small fragments according to preset rules and distributed and stored on different nodes. The system can process read and write requests of multiple data fragments in parallel, thereby significantly improving overall performance.

[0026] Determine the size of each shard based on the total amount of data and node capacity; divide the data according to business logic, such as timestamp, geographic location, user ID, etc.; hash sharding uses a hash function to map the unique identifier of the data to different shards.

[0027] Divide data into different shards according to the continuous range of a certain field; shard by list, assign data to specific shards according to predefined rules (such as alphabetical order, area code); consistent hash sharding, map data and nodes to a ring to ensure uniform distribution and dynamic scalability of data.

[0028] By splitting large BIM files into multiple small fragments and distributing and storing them on different nodes, the system can process read and write requests for multiple data fragments in parallel, thereby significantly improving overall performance. At the same time, combined with the load balancing algorithm, load balancing strategies such as polling, weighted polling, and minimum number of connections can be used to dynamically adjust request allocation according to the real-time load of the node, avoid performance bottlenecks caused by overloading a single node, and achieve balanced load distribution. At the same time, it can intelligently distribute read and write requests to nodes with lower loads, avoiding performance bottlenecks caused by overloading a single node.

[0029] 1.3 Data encryption and privacy protection Considering that BIM data may contain sensitive information, the present invention introduces a data encryption mechanism in the data storage process. Advanced encryption algorithms, such as AES-256, are used to encrypt BIM data. AES-256 is a widely recognized strong encryption algorithm that can provide a very high level of data protection.

[0030] Step 1: Before data storage, the system will encrypt the BIM data. The encryption process can be performed before data fragmentation, or each data fragment can be encrypted separately, depending on system design and performance considerations.

[0031] Step 2: The encrypted data will be stored on distributed nodes to ensure security during storage and transmission, effectively preventing data leakage and illegal access.

[0032] Step 3: Fine-grained access control policies ensure that project team members can only access data that matches their permissions, further enhancing data privacy protection.

[0033] This dual protection mechanism provides an extremely high level of security for BIM data, allowing users to safely use the system of the present invention to store and access their sensitive data.

[0034] 2. Read and write path optimization In terms of read and write path optimization, the present invention not only focuses on improving data access speed, but also focuses on reducing system resource consumption and improving user experience.

[0035] 2.1 Innovation of Intelligent Caching Strategy Traditional cache strategies are often based on simple LRU (Least Recently Used) or LFU (Least Frequently Used) algorithms. In order to further improve cache efficiency, this invention introduces an intelligent cache strategy based on machine learning. The specific steps are as follows: Step 1: Data Collection and Analysis The system collects log information of users’ access to BIM data, including access time, access frequency, access mode, etc.

[0036] Use machine learning algorithms (such as time series analysis, clustering algorithms, etc.) to conduct in-depth analysis of user access patterns and identify frequently accessed data patterns and access trends.

[0037] Step 2: Intelligent prediction and caching Based on the analysis results, the system predicts which BIM data will be frequently accessed.

[0038] The data predicted to be frequently accessed is cached in the local cache in advance to reduce the number of accesses to the remote storage and reduce the access latency.

[0039] Step 3: Dynamic Adjustment and Optimization The system monitors the usage of the cache in real time, including the cache hit rate, the validity of the cache data, etc.

[0040] Dynamically adjust the cache strategy based on the monitoring results, such as adjusting the cache size and replacement algorithm, to ensure that the data in the cache is always the most valuable.

[0041] 2.2 Innovation of concurrent reading and writing technology In order to improve the concurrent reading and writing performance, the present invention adopts a combination of multi-threading and asynchronous IO technology. The specific steps are as follows: Step 1: Multithreading The system uses multi-threading technology to create multiple worker threads to handle concurrent read and write requests.

[0042] Each worker thread independently processes a read or write request, enabling parallel processing of requests and improving CPU utilization.

[0043] Step 2: Asynchronous IO operation The system uses asynchronous IO technology, allowing read and write operations to be performed in the background without blocking the execution of the main thread.

[0044] The main thread can continue to process other tasks or requests, improving the system's responsiveness and throughput.

[0045] Step 3: Resource coordination and management The system coordinates and manages resources for multi-threaded and asynchronous IO operations to ensure reasonable allocation and efficient use of resources.

[0046] Through technical means such as thread pool and IO reuse, the system overhead is reduced and the concurrent processing capability is improved.

[0047] 2.3 Read-Write Conflict Resolution Mechanism In a distributed system, multiple users or processes may access the same BIM data at the same time. To solve this problem, the present invention designs an efficient read-write conflict resolution mechanism. The specific implementation steps are as follows: Step 1: Concurrency Control Strategy The system introduces concurrency control strategies such as optimistic locking or pessimistic locking to coordinate and manage concurrent read and write operations.

[0048] Optimistic locking is suitable for scenarios with more reads than writes, and uses version numbers or timestamps to detect whether data has been modified during reading and writing.

[0049] Pessimistic locking is suitable for scenarios with a lot of writes. It uses a locking mechanism to ensure that data is not interfered with by other operations when being written.

[0050] Step 2: Data consistency and integrity assurance The system ensures data consistency and integrity in a concurrent environment.

[0051] Ensure the accuracy and reliability of data through transaction management, log recording and other means.

[0052] Step 3: Exception handling and recovery mechanism The system supports transaction rollback and recovery mechanisms.

[0053] When abnormal situations occur, such as data inconsistency caused by read-write conflicts and system crashes, the system can roll back transactions in time and restore data to the correct state.

[0054] 3. Data Index Construction In order to quickly and accurately retrieve BIM data, the present invention establishes a multi-level and multi-dimensional data index system.

[0055] 3.1 Combination of static index and dynamic index Static indexes are mainly constructed based on the structural characteristics and metadata information of BIM data.

[0056] Step 1: During the data warehousing phase, the system creates a static index entry for each data segment and stores it in the index database.

[0057] Step 2: These static index entries support basic query operations, such as finding data segments based on ID or metadata information.

[0058] Step 3: Dynamic indexing dynamically adjusts the index structure based on user access patterns and data usage.

[0059] Step 4: The system automatically identifies frequently accessed data segments by monitoring user query requests and data access frequency, and promotes them to the top level of the index tree or a higher priority position.

[0060] This dynamic indexing mechanism can significantly improve query efficiency and reduce query latency.

[0061] 3.2 Fusion of spatial index and temporal index In view of the spatial and temporal characteristics of BIM data, the present invention also introduces spatial indexing and temporal indexing technologies. Spatial indexing supports spatial location-based queries by dividing spatial regions and establishing index entries for each region; temporal indexing builds indexes based on the timestamp information of BIM data to support time series analysis. The integration of these two indexing technologies enables users to query and analyze BIM data more flexibly.

[0062] 3.3 Index Maintenance and Optimization In order to maintain the real-time and accuracy of the index, the system also designs an index maintenance and optimization mechanism. When BIM data changes (such as adding, modifying or deleting data fragments), the system will synchronously update the index information to ensure the correctness of the query results. At the same time, the system will regularly evaluate the performance and efficiency of the index and optimize the index based on the evaluation results. For example, by merging sparse index entries, deleting invalid index entries or rebuilding the index tree, the query efficiency and space utilization of the index can be improved.

[0063] 4. Data Synchronization and Consistency In a distributed system, ensuring data consistency and synchronization is a crucial task. Aiming at the high real-time and high reliability requirements of BIM data, the present invention has made in-depth optimization and innovation in data synchronization and consistency.

[0064] 4.1 Combination of incremental synchronization and full synchronization In order to improve the efficiency of data synchronization, the present invention adopts a combination of incremental synchronization and full synchronization. Under normal circumstances, the system only synchronizes the data that has changed since the last synchronization through the incremental synchronization mechanism, thereby greatly reducing the amount of data transmission and shortening the synchronization time. However, in certain circumstances (such as system upgrades, node failure recovery, etc.), the system may need to perform full synchronization to ensure that the data on all nodes is completely consistent. This flexible synchronization strategy enables the system to select the most appropriate synchronization method based on actual needs.

[0065] 4.2 Log-based replication mechanism In order to achieve efficient incremental synchronization, the present invention introduces a log-based replication mechanism. On each storage node, the system maintains an operation log to record all modification operations (such as addition, deletion, and modification) on BIM data. When data synchronization is required, the system only needs to transfer the change records in the operation log to the target node and replay these change records on the target node to achieve data synchronization. This log-based replication mechanism not only improves synchronization efficiency, but also ensures the integrity and consistency of data during transmission.

[0066] 4.3 Conflict Detection and Resolution Mechanism In a distributed system, data conflicts may occur due to network delays, node failures, and other reasons. In order to solve this problem, the present invention designs an efficient conflict detection and resolution mechanism. When multiple nodes modify the same BIM data at the same time, the system detects potential conflicts by comparing the change records in the operation log. Once a conflict is found, the system automatically resolves the conflict according to the preset conflict resolution strategy (such as timestamp priority, version control, etc.) to ensure data consistency and accuracy.

[0067] 4.4 Real-time optimization In order to meet the high real-time requirements of BIM data, the present invention introduces real-time optimization measures in the data synchronization process. For example, the synchronization delay can be shortened by optimizing the network communication protocol, reducing the data verification and error processing time in the synchronization process, and improving the concurrent performance of log processing. In addition, the system also supports priority synchronization of BIM data with high real-time requirements to ensure that these data can be kept consistent among multiple nodes as soon as possible.

[0068] 4.5 Selection and Implementation of Consistency Model In a distributed system, the choice of consistency model is crucial to ensure data consistency and system performance. The present invention selects a suitable consistency model (such as eventual consistency, linear consistency, etc.) according to the characteristics and application scenarios of BIM data, and implements the model through a series of technical means. For example, by introducing distributed locks, transaction coordinators and other mechanisms to ensure the consistency and atomicity of cross-node operations; by optimizing data replication and synchronization strategies to reduce the risk of data inconsistency; by regularly performing consistency checks and repair operations to ensure data consistency in the long-term operation of the system.

[0069] In summary, the present invention has made in-depth optimization and innovation in data synchronization and consistency, ensuring the efficiency, reliability and consistency of BIM data in distributed systems. These technical means not only improve the overall performance and availability of the system, but also provide users with more stable and reliable BIM data storage and access services.

[0070] Beneficial effects of the present invention: Technical effects: 1) Significantly improve data processing capabilities The present invention stores BIM data in multiple nodes through the design of distributed architecture, which significantly improves the storage and processing capabilities of data. Traditional centralized storage solutions often have read and write performance bottlenecks and single point failure risks when facing large-scale BIM data. The present invention effectively alleviates these problems through data sharding and load balancing technology, allowing the system to efficiently process massive BIM data.

[0071] 2) Efficient data access and retrieval By constructing a multi-level and multi-dimensional data index system, the present invention realizes efficient access and retrieval of BIM data. The combination of static index and dynamic index, and the fusion of spatial index and time index enable users to quickly locate target data according to different query requirements, significantly improving query efficiency. At the same time, the application of intelligent caching strategy further reduces data reading delay and improves user experience.

[0072] 3) Strengthen data consistency and synchronization In terms of data synchronization and consistency, the present invention adopts a strategy combining incremental synchronization with full synchronization, as well as a log-based replication mechanism, to ensure efficient synchronization and consistency of data between multiple storage nodes. Through the conflict detection and resolution mechanism and the selection and implementation of the consistency model, the present invention effectively avoids data conflicts and inconsistencies, and ensures the high availability and reliability of BIM data in distributed systems.

[0073] 4) Enhance data security and privacy protection The present invention introduces a data encryption mechanism in the data storage process, and uses advanced encryption algorithms to encrypt BIM data, ensuring the security of data during storage and transmission. At the same time, the fine-grained access control mechanism allows project team members to access corresponding BIM data according to their permission levels, effectively preventing the risk of data leakage and illegal access.

[0074] Economic effect: 1) Reduce operation and maintenance costs Through distributed architecture and intelligent management strategies, the present invention realizes the automatic storage, access and synchronization of BIM data, reduces the need for manual intervention, and thus reduces operation and maintenance costs. At the same time, the system supports automatic expansion and contraction of nodes, and can automatically adjust resource allocation according to the dynamic changes in data volume, avoiding resource waste and cost increase.

[0075] 2) Improve project management efficiency Efficient BIM data storage and access methods can significantly improve project management efficiency. Project team members can access and share BIM data in real time, supporting real-time decision-making and collaborative work. This not only speeds up project progress, but also improves project quality and customer satisfaction, bringing greater economic benefits to the company.

[0076] 3) Promote technological innovation and industrial upgrading The implementation of this invention will promote the widespread application and innovative development of BIM technology in the fields of architecture, engineering, construction, etc. By improving the processing capacity and utilization efficiency of BIM data, this invention will promote the digital transformation and industrial upgrading of related industrial chains, and create more business opportunities and market value for enterprises.

[0077] Social impact: 1) Promoting sustainable development Efficient BIM data storage and access methods help promote the sustainable development of the construction industry. By sharing and analyzing BIM data in real time, project teams can more accurately assess resource consumption and environmental impact, develop more scientific and reasonable construction plans and resource allocation plans, and reduce resource waste and environmental pollution.

[0078] 2) Improve building safety and durability BIM technology provides more accurate and comprehensive information support for building design and construction management. Through the implementation of this invention, the project team can have a deeper understanding and analysis of the building structure and performance characteristics, timely discover and solve potential safety hazards and performance problems, and improve the safety and durability of the building.

[0079] 3) Promote the digital transformation of society The implementation of this invention will promote the digital transformation and intelligent development of various fields of society. By improving data processing capabilities and utilization efficiency, this invention will promote the deep integration and innovative application of advanced technologies such as digital technology and artificial intelligence in fields such as architecture, engineering, and construction, and bring a more intelligent, convenient, and efficient service experience to society. At the same time, the successful application of this invention will also provide useful experience and reference for the digital transformation of other industries.

[0080] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for efficient storage and access of BIM data based on a distributed architecture, characterized in that: The steps include: S1. Data distributed storage design: A highly scalable and fault-tolerant distributed file system is used as the basis to distribute BIM data to multiple storage nodes: S2. Read path optimization: to increase data access speed, reduce system resource consumption and improve user experience; S3. Establish a multi-level and multi-dimensional data index system to quickly and accurately retrieve BIM data; S4. Optimize data synchronization and consistency to meet the high real-time and high reliability requirements of BIM data.

2. The method for efficiently storing and accessing BIM data based on a distributed architecture according to claim 1, characterized in that: The distributed file system adopts Hadoop Distributed File System or open source Ceph distributed storage system.

3. The BIM data efficient storage and access method based on distributed architecture according to claim 1 is characterized in that: The data distributed storage design includes node selection and configuration optimization, data sharding and load balancing, and data encryption and privacy protection; The node selection and configuration optimization adopts an intelligent node selection and configuration strategy; firstly, the hardware resources and network bandwidth of each node are evaluated, and then according to the access characteristics and data volume of BIM data, storage nodes are dynamically selected for data storage; The data sharding and load balancing introduces data sharding technology, which splits large BIM files into multiple small fragments according to preset rules and distributes and stores them on different nodes; The data encryption and privacy protection uses an encryption algorithm to encrypt BIM data, providing an extremely high security level for BIM data, allowing users to store and access their sensitive data with confidence.

4. The method for efficiently storing and accessing BIM data based on a distributed architecture according to claim 3 is characterized in that: The specific implementation of data encryption and privacy protection is as follows: Step 1: Encrypt BIM data before data storage; the encryption process is performed before data fragmentation, or each data fragment is encrypted separately, depending on system design and performance considerations; Step 2: The encrypted data will be stored on distributed nodes to ensure security during storage and transmission, effectively preventing data leakage and illegal access; Step 3: Fine-grained access control policies ensure that project team members can only access data that matches their permissions, further enhancing data privacy protection.

5. The BIM data efficient storage and access method based on distributed architecture according to claim 1 is characterized in that: The read path optimization includes the innovation of intelligent cache strategy, innovation of concurrent read and write technology and read and write conflict resolution mechanism; The innovative implementation process of the intelligent caching strategy is as follows: Step 1: Data Collection and Analysis Collect log information of users accessing BIM data, including access time, access frequency, and access mode; Use machine learning algorithms to conduct in-depth analysis of user access patterns and identify frequently accessed data patterns and access trends; Step 2: Intelligent prediction and caching Based on the analysis results, predict which BIM data will be frequently accessed; Cache the data predicted to be frequently accessed in the local cache in advance to reduce the number of accesses to remote storage and reduce access latency; Step 3: Dynamic Adjustment and Optimization Real-time monitoring of cache usage, including cache hit rate and cache data validity; Dynamically adjust cache policies based on monitoring results, such as adjusting cache size and replacement algorithms, to ensure that the data in the cache is always the most valuable; The implementation process of the innovation of the concurrent reading and writing technology is as follows: Step 1: Multithreading Using multithreading technology, multiple worker threads are created to handle concurrent read and write requests; Each worker thread independently processes a read and write request, achieving parallel processing of requests and improving CPU utilization; Step 2: Asynchronous IO operation Using asynchronous IO technology, read and write operations are allowed to be performed in the background without blocking the execution of the main thread; The main thread continues to process other tasks or requests, improving the system's responsiveness and throughput; Step 3: Resource coordination and management Coordinate and manage resources for multi-threaded and asynchronous IO operations to ensure reasonable allocation and efficient use of resources; reduce system overhead and improve concurrent processing capabilities through thread pool and IO reuse technologies; The read-write conflict resolution mechanism is specifically as follows: Step 1: Concurrency Control Strategy Introduce optimistic locking or pessimistic locking concurrency control strategies to coordinate and manage concurrent read and write operations; Optimistic locking is suitable for scenarios with more reads than writes. It uses version numbers or timestamps to detect whether data has been modified during reading and writing. Pessimistic locking is suitable for scenarios with a lot of writes. It uses a locking mechanism to ensure that data is not interfered with by other operations when being written. Step 2: Data consistency and integrity assurance Ensure the consistency and integrity of data in a concurrent environment; ensure the correctness and reliability of data through transaction management and logging; Step 3: Exception handling and recovery mechanism Supports transaction rollback and recovery mechanisms; when an abnormal situation occurs, the transaction can be rolled back in time and the data can be restored to the correct state.

6. The method for efficiently storing and accessing BIM data based on a distributed architecture according to claim 1 is characterized in that: The establishment of a multi-level, multi-dimensional data index system includes the combination of static index and dynamic index, the fusion of spatial index and time index, and the maintenance and optimization of index; wherein, The specific implementation steps of combining the static index with the dynamic index are as follows: Step 1: During the data storage phase, a static index entry is created for each data segment and stored in the index database; Step 2: These static index entries support basic query operations; Step 3: Dynamic indexing dynamically adjusts the index structure based on user access patterns and data usage; Step 4: By monitoring the user's query requests and data access frequency, the frequently accessed data segments are automatically identified and promoted to the top level of the index tree or a position with a higher priority; The fusion of the spatial index and the time index is as follows: the spatial index supports queries based on spatial locations by dividing the spatial regions and establishing index entries for each region; the time index builds an index based on the timestamp information of the BIM data to support time series analysis; The maintenance and optimization of the index specifically includes: when the BIM data changes, the index information is synchronously updated to ensure the correctness of the query results; at the same time, the performance and efficiency of the index are regularly evaluated and the index is optimized and adjusted according to the evaluation results.

7. The method for efficiently storing and accessing BIM data based on a distributed architecture according to claim 1 is characterized in that: The optimization of data synchronization and consistency includes the combination of incremental synchronization and full synchronization, log-based replication mechanism, conflict detection and resolution mechanism, real-time optimization, and the selection and implementation of consistency model; The efficiency of data synchronization is improved by combining the incremental synchronization with the full synchronization; The log-based replication mechanism specifically means: on each storage node, an operation log is maintained to record all modification operations on BIM data; when data synchronization is required, the change records in the operation log are simply transferred to the target node, and these change records are replayed on the target node to achieve data synchronization; The conflict detection and resolution mechanism specifically refers to: when multiple nodes modify the same BIM data at the same time, potential conflicts are detected by comparing the change records in the operation log; once a conflict is found, the conflict is automatically resolved according to the preset conflict resolution strategy to ensure the consistency and accuracy of the data; The real-time optimization specifically refers to: shortening the synchronization delay by optimizing the network communication protocol, reducing the data verification and error processing time during the synchronization process, and improving the concurrent performance of log processing; The selection and implementation of the consistency model specifically refers to: ensuring the consistency and atomicity of cross-node operations by introducing distributed locks and transaction coordinator mechanisms; reducing the risk of data inconsistency by optimizing data replication and synchronization strategies; and ensuring data consistency in the long-term operation of the system by regularly performing consistency checks and repair operations.

Images