A construction data analysis and processing method and system

Abstract

This application provides a construction data analysis and processing method and system, relating to the technical field of data analysis. The method includes: acquiring the target construction data of the current construction project; dividing the construction schedule into multiple construction stages and dividing time windows according to the planned duration; constructing a hierarchical storage structure for each time window according to data type, and adding index tags containing project coordinates, construction time, and construction stage codes; storing data blocks into a distributed database based on the index tags, and generating a data lineage graph recording the flow path of the target construction data. This application improves the query efficiency of construction data.

Description

Technical Field

[0001] This application relates to the technical field of data analysis, and in particular to a method and system for analyzing and processing construction data. Background Technology

[0002] In today's rapidly developing construction industry, the quality and efficiency of building decoration construction are crucial to the success of the entire construction project. With the continuous expansion of building decoration projects, the increasing complexity of designs, and the tightening of construction cycles, massive amounts of data are generated during the construction process. This data covers all aspects from construction progress, material usage, and personnel allocation to quality inspection. Effective analysis of building decoration construction data can help construction teams identify problems in a timely manner, such as schedule delays, material waste, and substandard quality. This allows for targeted adjustments and optimizations, improving construction efficiency, reducing costs, and ensuring project quality.

[0003] Currently, some related technologies involve simple collection and preliminary statistics of construction data. For example, data is recorded manually or in simple spreadsheets, and then basic summarization and comparative analysis are performed to understand the general progress of the construction. Other technologies employ database management systems to store construction data, utilizing the database's query functions to retrieve and analyze specific data, providing some data support for construction management.

[0004] However, since construction data comes in many forms, the aforementioned technologies are relatively simple and crude in their storage methods. When it is necessary to query detailed data of a specific construction stage within a certain time period, it is often necessary to perform tedious filtering on a large amount of data, which wastes a lot of time and manpower. Summary of the Invention

[0005] To improve the efficiency of construction data retrieval, this application provides a construction data analysis and processing method and system.

[0006] Firstly, this application provides a method for analyzing and processing construction data, employing the following technical solution:

[0007] A method for analyzing and processing construction data includes the following steps:

[0008] Obtain the target construction data for the current construction project, divide the construction schedule into multiple construction stages, and divide the time window according to the construction schedule duration of each stage;

[0009] Within each time window, a hierarchical storage structure is constructed according to the type of target construction data. Index tags are added to the data blocks in the hierarchical storage structure. The index tags include project coordinates, construction time, and construction process codes.

[0010] Based on index tags, data blocks in the hierarchical storage structure are stored in a distributed database, and a data lineage graph is generated to record the flow path of target construction data.

[0011] This application achieves refined management of construction progress by dividing the construction schedule into multiple construction stages and further dividing it into time windows according to the planned duration, which helps to more accurately track the progress of each construction stage. Then, within each time window, a hierarchical storage structure is constructed according to the type of target construction data. This structured storage method makes the data more orderly, facilitating subsequent querying, analysis, and utilization. At the same time, hierarchical storage also improves the efficiency and flexibility of data storage.

[0012] This application adds index tags to data blocks in a hierarchical storage structure, including project coordinates, construction time, and construction stage codes. This provides a multi-dimensional retrieval approach for the data, enabling users to quickly locate the required data blocks according to different needs, thus improving the efficiency and accuracy of data retrieval. Through index tags, users can easily access data blocks in the distributed database, which helps improve the informatization level of construction projects and promotes data sharing and collaborative work among departments.

[0013] This application also generates a data lineage diagram for recording the flow path of target construction data. The generation of the data lineage diagram makes the source, destination and flow process of the data clearly visible. When problems occur or audits are required, the relevant data can be quickly located and its flow path can be traced.

[0014] Optionally, the method further includes:

[0015] Virtual links are established between data blocks that belong to different time windows and are related, and the virtual links are marked in the data lineage graph. Dynamic weight factors are embedded in the virtual links, and the dynamic weight factors are automatically adjusted based on the dependency relationship between construction stages and the data block call frequency.

[0016] When a data block is invoked, data blocks associated with that data block are preloaded in descending order of dynamic weight factors.

[0017] This application breaks the limitation of time windows on data association by establishing virtual links between data blocks belonging to different time windows and having a relationship. This makes the data of the entire construction project logically form a tighter whole, thereby more accurately reflecting the actual relationship between data at different construction stages and different time points in the construction project, and helping to deeply understand the internal logic of the construction process.

[0018] This application enriches the display dimensions of data flow by marking virtual links in the data lineage graph. The original data lineage graph recorded the data generation and flow paths, while the marking of virtual links allows the graph to present more complex relationships between data. This application also embeds dynamic weighting factors into the virtual links, and these factors can be automatically adjusted based on the dependencies between construction stages and the frequency of data block calls, making data management more flexible and intelligent.

[0019] When a data block is accessed, related data blocks are preloaded in descending order of dynamic weight factors. This allows potentially relevant data to be loaded into memory in advance, reducing waiting time for subsequent data retrieval. Since the dynamic weight factor reflects the degree of correlation and importance between data blocks, preloading in weight order prioritizes loading data most closely related to the currently accessed data block and most likely to be used, thereby improving data targeting and query efficiency. The preloading mechanism effectively reduces system performance bottlenecks caused by frequent access to external storage devices such as disks. This application reduces disk I / O operations and improves data query response speed by preloading data into memory. Simultaneously, the preloading method guided by dynamic weight factors reduces the risk of blindly preloading large amounts of irrelevant data, lowers memory resource consumption, and allows for more efficient use of memory resources.

[0020] By adopting the above solution, this application provides construction project managers with more timely and accurate data support, enabling them to acquire and use data more efficiently and smoothly, reducing communication costs and work delays caused by untimely or inaccurate data acquisition, and improving the overall collaborative efficiency of the construction project.

[0021] Optionally, the virtual link is generated as follows:

[0022] Using the Geohash algorithm and project coordinates, a geohash code is generated. A 256-bit hash value is generated based on the project ID, geohash code, and time window number. The first 16 characters of the hash value are extracted, and a 4-bit random salt is added after the 16 characters to obtain an identifier. The identifier is used as the primary key value of the data block, and a virtual link is established using the primary key value.

[0023] This application utilizes the Geohash algorithm and project coordinates to generate geohash codes, which can convert specific geographic coordinate information into codes with a certain degree of precision, achieving an accurate representation of the project's geographical location. In construction projects, construction stages and data at different geographical locations may have specific correlations. The above scheme helps to accurately identify and associate geographically related data blocks, providing a foundation for the accuracy of establishing virtual links on a geographic dimension.

[0024] Subsequently, this application generates a 256-bit hash value based on the project ID, geohash code, and time window number, which can more comprehensively and accurately describe the relationship between different data blocks in the construction project, so that the generated virtual link can accurately reflect the logical connection in the actual construction process.

[0025] This application obtains an identifier by extracting the first 16 characters of a 256-bit hash value and adding a 4-bit random salt afterward. This enhances the uniqueness of the identifier, minimizes duplicate identifiers, and ensures that each virtual link has a unique identifier, reducing data conflicts and incorrect associations. The added 4-bit random salt adds randomness and uncertainty to the identifier. Faced with potential security threats, attackers find it difficult to obtain the original input information or generate the same identifier through simple reverse engineering or guessing. The presence of the random salt makes the identifier generation process more complex and unpredictable, improving security and protecting sensitive data in construction projects from unauthorized acquisition and tampering.

[0026] Using the generated identifier as the primary key value of the distributed database, the operation of establishing virtual links using the primary key value is relatively convenient. In the distributed database, related data blocks can be quickly located and associated through the primary key value. The process of establishing virtual links can be completed in a short time, which improves the efficiency of data association and management and helps to quickly query and process related data.

[0027] Optionally, the generation process of the data kinship map is as follows:

[0028] The initial graph is constructed by treating data blocks as nodes, the index labels of data blocks as node attributes, and the flow paths of data blocks between construction stages as directed edges.

[0029] Based on the hierarchical storage structure to which the data blocks belong, the initial graph is processed into hierarchical layers to obtain a data lineage graph.

[0030] This application uses data blocks as nodes and the flow paths of data blocks between construction stages as directed edges to construct an initial graph. In a construction project, the target construction data will continuously change and be transmitted as different construction stages progress. Directed edges can clearly show which stage the data flows from to which stage. For example, in building decoration engineering, concrete mix proportion data may flow from the design stage to the construction stage, and then, after adjustments based on actual construction feedback, flow back to the design stage for optimization. Directed edges can accurately represent this process, making it easier for relevant personnel to understand the dynamic changes of data in the project.

[0031] This application uses the index labels of data blocks as node attributes, endowing each data block with rich background information. The index labels record in detail the location, time, and relevant stage of the target construction data. When it is necessary to trace the source or destination of certain data, the relevant construction stage and time point can be quickly located by viewing the node attribute information, improving the efficiency and accuracy of data tracing. Moreover, the data lineage graph supports data tracing from multiple dimensions, which helps to promptly identify and resolve data-related issues.

[0032] This application also performs hierarchical processing on the initial graph based on the hierarchical storage structure to which the data blocks belong, so that the data lineage graph matches the actual data storage method. The hierarchical storage structure can be divided according to factors such as data type, importance, or usage frequency. By processing the initial graph hierarchically, the relationships between data at different levels can be more clearly displayed, which helps to optimize data storage and management, and improve data storage efficiency and maintainability.

[0033] Optionally, after generating the data kinship map, the method further includes:

[0034] Develop verification rules based on the type of target construction data, and mark key links in all construction stages;

[0035] Validation rules are embedded in the data lineage graph. When a data block is transferred to a critical stage, the validation rules are used to validate the data block, obtain the validation results, and mark abnormal data in the data lineage graph based on the validation results.

[0036] This application establishes verification rules based on the type of target construction data. These rules are tailored to the specific type and characteristics of the construction data, enabling precise detection of whether the target construction data meets the expected format, range, logic, and other requirements. For example, for concrete strength data, the verification rules can stipulate that its value must be within a specific strength grade range. Verifying the data when it flows to the corresponding construction stage can promptly identify errors or anomalies in the data, minimizing the propagation and amplification of erroneous data in subsequent construction stages, thereby significantly improving the accuracy of the construction data.

[0037] This application achieves full-process monitoring of construction data flow by marking key stages in all construction phases and employing verification rules to validate the data blocks of these key stages. Key stages are typically those where data quality is prone to problems or have a significant impact on subsequent construction. By focusing on verifying the target construction data of key stages, this application can improve the legitimacy of the data.

[0038] This application improves the efficiency and timeliness of data verification by embedding verification rules into the data lineage graph and automatically triggering the verification function when data blocks flow to key stages. Compared with traditional manual verification methods, automated verification can avoid interference and errors from human factors, and reduce the time and labor costs of verification work.

[0039] Optionally, the method further includes:

[0040] The system acquires real-time construction data of the current construction project, construction data curves of the current construction project, and historical construction data curves of multiple historical construction projects. Using the DTW algorithm, the system aligns the construction data curve of the current construction project with each historical construction data curve to obtain the historical construction data corresponding to the real-time construction data in each historical construction data curve, which is denoted as the target data. The target data and the remaining historical construction data at the same time as the target data are integrated into a construction vector according to a preset order.

[0041] All construction vectors are integrated into an evaluation matrix, and a multi-objective decision-making algorithm is used to determine risk points based on the evaluation matrix.

[0042] This application employs the DTW (Dynamic Time Warping) algorithm to align the current construction data curve of a project with each historical construction data curve, overcoming potential deviations in the timeline between different construction projects. In actual construction, due to various factors (such as weather, adjustments to construction schedules, etc.), the construction pace of different projects may differ. The DTW algorithm can dynamically adjust the timeline matching, ensuring accurate correspondence between the current real-time construction data and the corresponding stage data in the historical construction data.

[0043] This application can efficiently process large amounts of real-time and historical construction data. By automating the alignment and integration operations through algorithms, it reduces the time and errors associated with manual data processing. In large-scale construction projects, massive amounts of construction data are generated daily, making manual data alignment and integration virtually impossible. This application, however, can complete these tasks quickly and accurately, improving the efficiency and accuracy of data processing.

[0044] This application integrates all construction vectors into an evaluation matrix, and uses a multi-objective decision-making algorithm based on the evaluation matrix to determine risk points, enabling a comprehensive analysis of construction risks from multiple dimensions. The evaluation matrix contains various construction data from different historical construction projects and current construction projects. The multi-objective decision-making algorithm can comprehensively consider the weights and interrelationships of these data, minimizing risk assessment bias caused by single data or single objectives.

[0045] Optionally, the step of determining risk points using a multi-objective decision-making algorithm based on the evaluation matrix includes: standardizing the elements in the evaluation matrix, obtaining positive and negative ideal solutions in the processed evaluation matrix, obtaining the relative proximity of each construction vector based on the positive and negative ideal solutions, recording the construction vector corresponding to the maximum relative proximity as the target vector, comparing the real-time construction data with the elements in the target vector, and determining the risk points based on the comparison results.

[0046] By standardizing the data, this application can unify data of different dimensions and ranges to a relatively consistent scale, making them comparable. Subsequent multi-objective decision analysis based on these data can be more accurate and reasonable, minimizing analytical biases caused by different dimensions.

[0047] This application obtains positive and negative ideal solutions from the processed evaluation matrix, constructing clear boundaries for risk identification and helping to more accurately determine the risk zone where the current construction data is located. Subsequently, based on the positive and negative ideal solutions, the relative proximity of each construction vector is obtained, which can accurately measure how close each construction vector (i.e., the combination of data under different construction states) is to the ideal state and the worst-case state. The greater the relative proximity, the closer the construction vector is to the positive ideal solution, and the lower the risk level; conversely, the smaller the relative proximity, the closer it is to the negative ideal solution, and the higher the risk level.

[0048] Subsequently, this application denotes the construction vector corresponding to the highest relative closeness as the target vector. This allows for the identification of the closest possible state to the ideal state during the current construction phase. Although the target vector itself represents a relatively good state, comparing it with real-time construction data can more accurately identify deviations between the real-time data and the ideal state. These deviations often represent potential risk points. For example, in architectural decoration construction, the target vector may correspond to high levels of indicators such as wall flatness and color uniformity, while real-time construction data may show flatness deviations or color unevenness in certain areas of the wall. By comparing these deviations, the potential quality risks can be accurately identified.

[0049] This application identifies risk points based on comparison results, providing clear direction for construction management decisions. Construction managers can formulate corresponding risk response strategies and measures for different risk points. For example, if the comparison results show that the concrete strength data of a certain construction stage is lower than the value in the target vector, indicating a quality risk, then managers can decide to increase the frequency of concrete testing for that stage, adjust the concrete mix ratio, or reinforce the already constructed parts. Decisions based on clearly identified risk points in the above schemes are more targeted, improving the effectiveness and efficiency of decision-making.

[0050] Optionally, the method further includes:

[0051] Obtain historical construction data for each construction project, use the historical construction data to train the pre-built prediction model, obtain the trained prediction model, input all construction data of the current construction project into the trained prediction model, and output the predicted construction data.

[0052] Based on the predicted construction data and all construction data, a predicted construction data curve is plotted. The predicted construction data curve is then updated to the construction data curve of the current construction project, and the predicted construction data is updated to the real-time construction data.

[0053] This application acquires historical construction data for each construction project and uses this data to train a pre-built predictive model, thereby fully uncovering the patterns and regularities within the historical data. Through training with a large amount of historical data, the predictive model can learn the inherent relationships and trends in the construction process, thus improving the accuracy of predicting future construction conditions.

[0054] All construction data of the current construction project are input into the trained prediction model, which outputs predicted construction data. A predicted construction data curve is plotted based on the predicted construction data and all construction data. The predicted construction data curve is then updated to the construction data curve of the current construction project. The predicted construction data is then updated to the real-time construction data. After that, the evaluation model is reconstructed based on the above construction data curve and real-time construction data to analyze risk points and achieve risk prediction.

[0055] Optionally, after integrating all construction vectors into an evaluation matrix, the method further includes:

[0056] Based on each column element in the evaluation matrix, calculate the first quartile, the third quartile, and the interquartile range. Set the value range of the i-th column element according to the first quartile, the third quartile, and the interquartile range. Traverse the i-th column element. If there is an element that does not belong to the value range, determine that the element is an outlier.

[0057] For the identified outliers, the deviation between the outlier and the range of values ​​is calculated, and it is determined whether the deviation is greater than a preset deviation threshold. If so, the outlier is analyzed and processed according to the analysis results. If not, the mean of the remaining elements in the column where the outlier is located is calculated, and the mean is used to replace the outlier.

[0058] This application calculates the first quartile, third quartile, and interquartile range based on each column element in the evaluation matrix, and sets the value range of the i-th column element according to these statistics. This can more scientifically define the fluctuation range of normal data and more accurately identify outliers that deviate from the main distribution of the data compared to the simple mean and standard deviation method.

[0059] By traversing the elements in the i-th column, each element is checked to see if it falls within the set value range. If not, the element is marked as an outlier. For the identified outliers, the deviation from the normal range is calculated, and it is determined whether the deviation is greater than a preset deviation threshold. For outliers with large deviations, it indicates that the difference from the normal data is significant, which may indicate serious problems such as data entry errors or the influence of special events. In-depth analysis and processing based on the analysis results are required. For outliers with small deviations, it may be due to data measurement errors or accidental situations within the normal fluctuation range. The mean of the remaining elements in the column is used to replace the outlier, which can eliminate the impact of outliers on the overall data and ensure the relative rationality of the data.

[0060] Secondly, this application provides a construction data analysis and processing system, which adopts the following technical solution:

[0061] A construction data analysis and processing system includes: a processor, and a memory communicatively connected to the processor;

[0062] The memory stores a computer-readable storage medium, and the computer-readable storage medium stores a computer program.

[0063] When the processor processes a computer program stored on the computer-readable storage medium, it implements the method as described in the first aspect.

[0064] In summary, this application includes at least one of the following beneficial technical effects:

[0065] 1. This application achieves refined management of construction progress by dividing the construction schedule into multiple construction stages and further dividing it into time windows according to the planned duration, which helps to more accurately track the progress of each construction stage. Then, within each time window, a hierarchical storage structure is constructed according to the type of target construction data. This structured storage method makes the data more orderly, facilitating subsequent querying, analysis, and utilization. At the same time, hierarchical storage also improves the efficiency and flexibility of data storage.

[0066] 2. This application adds index tags to data blocks in the hierarchical storage structure, including project coordinates, construction time, and construction stage codes, providing a multi-dimensional retrieval approach for the data. This allows users to quickly locate the required data blocks according to different needs, improving the efficiency and accuracy of data retrieval. Through index tags, users can easily access data blocks in the distributed database, which helps improve the informatization level of construction projects and promotes data sharing and collaborative work among departments.

[0067] 3. This application also generates a data lineage diagram for recording the flow path of target construction data. The generation of the data lineage diagram makes the source, destination and flow process of the data clearly visible. When problems occur or audits are required, the relevant data can be quickly located and its flow path can be traced. Attached Figure Description

[0068] Figure 1 This is a flowchart of Embodiment 1 of this application;

[0069] Figure 2 This is a flowchart of Embodiment 3 of this application. Detailed Implementation

[0070] The target construction data, real-time construction data, historical construction data, and all construction data mentioned in this application include: progress data, resource data, and environmental data. The progress data includes the actual amount completed and the planned amount completed. The resource data includes manpower data, mechanical construction rate, and material consumption. The environmental data includes temperature, humidity, and dust concentration.

[0071] The following combination Figure 1 and Figure 2 This application will be described in further detail.

[0072] Example 1: This example discloses a construction data analysis and processing method, referring to... Figure 1 The method includes: S11 data acquisition, S12 constructing a hierarchical storage structure, and S13 storage. This involves acquiring the target construction data for the current construction project, dividing the construction schedule into multiple construction phases and dividing time windows according to the planned duration; constructing a hierarchical storage structure for each time window based on data type, and adding index tags containing project coordinates, construction time, and construction phase codes; storing data blocks in a distributed database based on the index tags, and generating a data lineage graph recording the flow path of the target construction data. The execution process of each step in this embodiment is as follows:

[0073] S11 Data Acquisition: Obtain the target construction data for the current construction project. The target construction data includes: progress data, resource data, and environmental data. The progress data includes the actual completed amount and the planned completed amount. The resource data includes manpower data, mechanical construction rate, and material consumption. The environmental data includes temperature, humidity, and dust concentration.

[0074] Based on the WBS (Work Breakdown Structure), the construction schedule is divided into multiple construction phases, and time windows are defined according to the planned duration of each construction phase.

[0075] S12 constructs a hierarchical storage structure. Within each time window, a hierarchical storage structure is constructed according to the type of target construction data. In this embodiment, the hierarchical storage structure is constructed according to the type of target construction data, including: progress data layer, manpower data layer, mechanical construction rate layer, material consumption layer, temperature layer, humidity layer, and dust concentration layer.

[0076] In other embodiments, the hierarchical storage structure includes a raw layer for storing raw acquired data, a feature layer for storing data features, and an aggregation layer for storing processing results.

[0077] Add index tags to the data blocks in the hierarchical storage structure. The index tags include project coordinates, construction time and construction process codes. The project coordinates refer to the global coordinates in the BIM model. The construction time refers to the timestamp and time window number of the data acquisition. The construction process codes are coded according to the coding rules in GB / T 51301-2018 "Standard for Delivery of Building Engineering Design Information Model".

[0078] In the hierarchical storage structure, a data block refers to dividing target structure data belonging to the same time window and the same layer into a single data block.

[0079] S13 storage converts index tags into metadata fields for a distributed database, creating composite indexes such as spatiotemporal indexes (e.g., time window + timestamp) and spatial task indexes (e.g., component ID + stage code) to accelerate typical queries in construction scenarios. For example, it allows for quick retrieval of construction data for a specific column during a specific time period using component ID and time window indexes, or analysis of equipment utilization trends using machine ID and timestamps.

[0080] The spatiotemporal index uses time windows as the primary index and timestamps as the secondary index, forming a hierarchical relationship of time window → timestamp → data block. The construction method is as follows: the target construction data is physically fragmented according to time windows (e.g., each time window corresponds to a storage volume). When the target construction data is written, the time window and timestamp are automatically extracted to generate a composite index key and stored in the index table.

[0081] When querying all data within a specific time period, the primary index directly locates the shard of the target time window. When querying detailed data for a specific moment, the secondary index quickly locates the corresponding construction data.

[0082] The spatial task index uses component ID as the primary index and construction process code as the secondary index to form a hierarchical relationship of component ID → construction process code → data block. The construction method is as follows: during data acquisition, the component ID is parsed through the BIM model, and the construction process code is extracted from the construction plan to generate a composite index key. The mapping relationship between the composite index key and the data block storage path is stored in a relational database.

[0083] When you need to query all task data for a certain component, you can directly locate it through the component ID index. When you need to query all components involved in a certain construction stage, you can retrieve them in reverse through the stage code index.

[0084] By combining spatiotemporal indexes and spatial task indexes, complex query scenarios can be supported. For example:

[0085] We need to check the concrete pouring progress of column COL-003-2F between 14:00 and 16:00 on March 15, 2024.

[0086] The query path is as follows: locate the W20240315_1400 fragment through the spatiotemporal index; within the W20240315_1400 fragment, retrieve the target data block through the spatial task index COL-003-2F:P001.

[0087] During data storage, the source information, transformation logic, dependencies, and other lineage information of the target construction data are recorded synchronously. Source information records the original origin of each piece of construction data, such as specific sensor numbers, manual entry form names, and third-party system interface addresses. Transformation logic clarifies the processing rules for data flow, including calculation formulas or data cleaning rules. Dependencies are used to establish relationships between data blocks; for example, a pouring schedule depends on two preceding data blocks: material arrival records and personnel scheduling records.

[0088] By treating data blocks as nodes, their index labels as node attributes, and the flow paths of data blocks between construction stages as directed edges, an initial graph is constructed.

[0089] Based on the hierarchical storage structure to which the data blocks belong, the initial graph is processed into hierarchical layers to obtain a data lineage graph.

[0090] The layering process for the initial map refers to placing data blocks belonging to the progress data layer, manpower data layer, mechanical construction rate layer, material consumption layer, temperature layer, humidity layer, or dust concentration layer into the corresponding map layer according to their storage level.

[0091] Validation rules are constructed for different types of target construction data. The validation rules in this embodiment include:

[0092] Format validation rules include: field type matching (e.g., age must be an integer), required field checks, and date format checks.

[0093] Legality verification rules: temperature value between -20℃ and 50℃, concrete strength greater than or equal to 30MPa, etc.

[0094] Consistency verification rules: Order IDs must be consistent in the order table and payment table, and device IDs must match the device types in the model library, etc.

[0095] Business logic verification rules: the completion time of rebar tying cannot be earlier than the foundation pouring time, and the concrete curing days must be greater than or equal to 7 days, etc.

[0096] Integrity verification rules: Daily construction logs must include fields such as worker_count and material_usage.

[0097] Mark the critical links in all construction stages. The critical links refer to the links that have a direct impact on construction safety, quality, progress and cost, such as concrete pouring, rebar tying and equipment installation.

[0098] A rule engine is used to abstract the validation logic in the validation rules into configurable rule scripts, which are then stored in a distributed database. New attribute sets are added to the nodes corresponding to the key links in the data lineage graph. These new attribute sets are used to embed validation rules. Subsequently, the applicable validation rules for the node are associated with the composite key of the composite index to build an inverted index.

[0099] When a data block reaches a critical stage, the associated verification rules are loaded through the composite key of the composite index. The verification rules are then used to verify the data block and obtain the verification results. If the verification results indicate an anomaly, the abnormal data is marked in the data lineage graph.

[0100] By adopting the above solution, this embodiment achieves structured storage, efficient retrieval, and end-to-end traceability of construction data. In particular, by layering data according to data type or data processing stage, this embodiment can minimize data clutter and ensure accurate location of the corresponding level during subsequent searches and retrievals.

[0101] Example 2: This example differs from Example 1 in that the method further includes:

[0102] Virtual links are established between data blocks belonging to different time windows and being related, and these virtual links are marked in the data lineage graph. The process of generating these virtual links is as follows:

[0103] The horizontal and vertical coordinates of the project are input into the Geohash algorithm to obtain a 6-bit calculation result. The vertical coordinate is then processed into binary encoding (i.e., the decimal vertical coordinate value is converted into binary) to obtain an 8-bit encoding value. The calculation result and the encoding value are concatenated to obtain a 14-bit geohash code, thereby encoding the spatial location into a string, which facilitates the association of regional data blocks.

[0104] Next, the project ID, geohash code, and time window number are concatenated into a string in sequence. The SHA-256 algorithm is used to perform a hash calculation on the concatenated string to obtain a 256-bit hash value. The first 16 characters of the hash value are extracted, and a 4-bit random salt value is appended to the end of the 16 characters to obtain an identifier. The identifier is used as the primary key value of the data block. Virtual links are established using the primary key value to form a logical virtual connection.

[0105] A dynamic weighting factor is embedded in the virtual link. This dynamic weighting factor is automatically adjusted based on the dependencies between construction stages and the frequency of data block calls. The calculation method of the dynamic weighting factor is as follows:

[0106] Dynamic weighting factor = Dependency weight × Dependency factor + Data block call frequency weight × Data block call frequency;

[0107] The dependency factor is calculated as follows:

[0108] ;

[0109] This represents the numerical value of the dependency factor; This represents the number of nodes from the i-th node (inclusive) to the j-th node (inclusive) in the virtual connection. This represents the number of nodes contained in the virtual connection.

[0110] The data block call frequency is equal to the ratio of the number of times the current data block is called to the total number of times all data blocks are called;

[0111] Dependency weight + data block call frequency weight = 1;

[0112] When a data block is invoked, data blocks associated with that data block are preloaded in descending order of dynamic weight factors.

[0113] This embodiment uses the Geohash algorithm and hash identifiers to uniquely mark data blocks across time windows, combines dynamic weight factors to quantify the importance of virtual links, and finally improves the efficiency of construction data access through a preloading mechanism.

[0114] Example 3: Reference Figure 2The difference between this embodiment and Embodiment 1 is that the method further includes:

[0115] S31 obtains the construction vector, acquires the real-time construction data of the current construction project, plots the construction data curve of the current construction project based on all the construction data of the current construction project, and plots the historical construction data curve of each historical construction project based on all the construction data of that historical construction project.

[0116] Using the DTW algorithm, the i-th historical construction data curve is analyzed. Dynamically time-warp the current construction data curve G(t) to calculate the optimal alignment path. Based on the alignment path, extract the historical construction data corresponding to the real-time construction data from the i-th historical construction data curve. , denoted as target data , target data And the remaining historical construction data at the same time as the target data in the same historical construction project. The data are integrated into construction vectors according to a preset order. The calculation model for the construction vectors is as follows:

[0117] ;

[0118] in, Let i be the construction vector corresponding to the i-th historical construction data curve; is the target data; n is the number of remaining historical construction data.

[0119] S32 obtains the evaluation matrix. In this embodiment, all construction vectors are stacked row by row to form the evaluation matrix. In other embodiments, all construction vectors can also be stacked column by column.

[0120] For each column of the evaluation matrix, calculate the first quartile Q1, the third quartile Q3, and the interquartile range IQR for each column element, where IQR = Q3 - Q1.

[0121] The value range of the element in the i-th column is determined by the first quartile, the third quartile, and the interquartile range, and the value range is [Q1-1.5*IQR, Q3+1.5*IQR].

[0122] Iterate through the elements in column i. If there is an element in column i that does not belong to the range of values, mark the element that does not belong to the range of values ​​as an outlier; otherwise, execute S33 to determine the risk point.

[0123] For identified outliers, the deviation between the outlier and the range of values ​​is calculated, and it is determined whether the deviation is greater than the preset deviation threshold. If so, the outlier is analyzed, that is, whether the outlier is modifiable. Modifiable situations include: input errors, sudden events, etc.

[0124] Outliers are processed based on the analysis results. For example, incorrectly entered data is updated to correct data, and the processed outliers are updated in the evaluation matrix. Alternatively, if the outlier is caused by a sudden event, the construction vector containing the outlier is removed from the evaluation matrix. Otherwise, the mean of the remaining elements in the column containing the outlier that are within the range of values ​​is calculated, and the mean is updated in the evaluation matrix.

[0125] S33 identifies risk points by using a multi-objective decision-making algorithm based on the evaluation matrix. This includes: standardizing elements belonging to the same category in the evaluation matrix (in this embodiment, since the construction vectors are stacked by row, the standardization is performed on elements in the same column; conversely, in other embodiments, the construction vectors are stacked by column, so the standardization is performed on elements in the same row). The positive ideal solution and the negative ideal solution are then obtained from the processed evaluation matrix. The positive ideal solution refers to each element in the construction vector reaching its optimal value, and the negative ideal solution refers to each element in the construction vector reaching its worst value.

[0126] The relative proximity of each construction vector is obtained based on the positive and negative ideal solutions, as follows:

[0127] Calculate the Euclidean distance between the k-th construction vector and the positive ideal solution in the evaluation matrix. Then calculate the Euclidean distance between the construction vector and the negative ideal solution. Finally, the relative proximity is calculated, and the calculation model for the relative proximity is as follows:

[0128] ;

[0129] in, This is the numerical value of the relative proximity of the k-th construction vector.

[0130] The construction vector corresponding to the maximum relative proximity is denoted as the target vector. The real-time construction data is compared with the elements in the target vector to obtain the deviation between the real-time construction data and the corresponding historical construction data in the target vector. The risk points are determined based on the deviation.

[0131] In this embodiment, determining the risk point based on the deviation means that when the negative deviation between a real-time construction data and historical construction data exceeds expectations (the negative deviation refers to project construction delays, material consumption exceeding historical construction projects, etc.), the real-time construction data is identified as having a risk, i.e., a risk point.

[0132] In other embodiments, in S31, when obtaining the construction vector, the DTW algorithm is used to process the i-th historical construction data curve. Before performing dynamic time warping on the current construction data curve G(t) and calculating the optimal alignment path, the method further includes:

[0133] Historical construction data for each construction project is obtained, and the pre-built prediction model is trained using the historical construction data to obtain the trained prediction model. The prediction model can be any one of the following models: convolutional neural network model, long short-term memory network model, etc.

[0134] Input all construction data of the current construction project into the trained prediction model, and output the predicted construction data.

[0135] Based on the predicted construction data and all construction data, a predicted construction data curve is plotted. The predicted construction data curve is then updated to the construction data curve of the current construction project, and the predicted construction data is updated to the real-time construction data.

[0136] By adopting the above solution, this embodiment achieves dynamic monitoring of construction risks.

[0137] Example 4: This example discloses a construction data analysis and processing system, including: a processor, and a memory communicatively connected to the processor;

[0138] The memory stores a computer-readable storage medium, and the computer-readable storage medium stores a computer program.

[0139] When the processor processes the computer program stored on the computer-readable storage medium, it implements the construction data analysis and processing method.

[0140] The above are all preferred embodiments of this application, and are not intended to limit the scope of protection of this application. Therefore, all equivalent changes made in accordance with the structure, shape and principle of this application should be covered within the scope of protection of this application.

Claims

1. A method for analyzing and processing construction data, characterized in that, include: Obtain the target construction data for the current construction project, divide the construction schedule into multiple construction stages, and divide the time window according to the construction schedule duration of each stage; Within each time window, a hierarchical storage structure is constructed according to the type of target construction data. Index tags are added to the data blocks in the hierarchical storage structure. The index tags include project coordinates, construction time, and construction process codes. Based on index tags, data blocks in the hierarchical storage structure are stored into a distributed database, and a data lineage graph is generated to record the flow path of target construction data. The generation process of the data kinship map is as follows: The initial graph is constructed by treating data blocks as nodes, the index labels of data blocks as node attributes, and the flow paths of data blocks between construction stages as directed edges. Based on the hierarchical storage structure to which the data blocks belong, the initial graph is processed into hierarchical layers to obtain a data lineage graph; Virtual links are established between data blocks that belong to different time windows and are related, and the virtual links are marked in the data lineage graph. Dynamic weight factors are embedded in the virtual links, and the dynamic weight factors are automatically adjusted based on the dependency relationship between construction stages and the data block call frequency. When a data block is invoked, data blocks associated with that data block are preloaded in descending order of dynamic weight factors.

2. The construction data analysis and processing method according to claim 1, characterized in that, The process of generating the virtual link is as follows: Using the Geohash algorithm and project coordinates, a geohash code is generated. A 256-bit hash value is generated based on the project ID, geohash code, and time window number. The first 16 characters of the hash value are extracted, and a 4-bit random salt is added after the 16 characters to obtain an identifier. The identifier is used as the primary key value of the data block, and a virtual link is established using the primary key value.

3. The construction data analysis and processing method according to claim 1, characterized in that, After generating the data kinship map, the method further includes: Develop verification rules based on the type of target construction data, and mark key links in all construction stages; Validation rules are embedded in the data lineage graph. When a data block is transferred to a critical stage, the validation rules are used to validate the data block, obtain the validation results, and mark abnormal data in the data lineage graph based on the validation results.

4. The construction data analysis and processing method according to claim 1 or 2, characterized in that, The method further includes: The system acquires real-time construction data of the current construction project, construction data curves of the current construction project, and historical construction data curves of multiple historical construction projects. Using the DTW algorithm, the system aligns the construction data curve of the current construction project with each historical construction data curve to obtain the historical construction data corresponding to the real-time construction data in each historical construction data curve, which is denoted as the target data. The target data and the remaining historical construction data at the same time as the target data are integrated into a construction vector according to a preset order. All construction vectors are integrated into an evaluation matrix, and a multi-objective decision-making algorithm is used to determine risk points based on the evaluation matrix.

5. The construction data analysis and processing method according to claim 4, characterized in that, The method of determining risk points using a multi-objective decision-making algorithm based on the evaluation matrix includes: standardizing the elements in the evaluation matrix, obtaining positive and negative ideal solutions in the processed evaluation matrix, obtaining the relative proximity of each construction vector based on the positive and negative ideal solutions, recording the construction vector corresponding to the maximum relative proximity as the target vector, comparing the real-time construction data with the elements in the target vector, and determining the risk points based on the comparison results.

6. The construction data analysis and processing method according to claim 4, characterized in that, The method further includes: Obtain historical construction data for each construction project, use the historical construction data to train the pre-built prediction model, obtain the trained prediction model, input all construction data of the current construction project into the trained prediction model, and output the predicted construction data. Based on the predicted construction data and all construction data, a predicted construction data curve is plotted. The predicted construction data curve is then updated to the construction data curve of the current construction project, and the predicted construction data is updated to the real-time construction data.

7. The construction data analysis and processing method according to claim 4, characterized in that, After integrating all construction vectors into an evaluation matrix, the method further includes: Based on each column element in the evaluation matrix, calculate the first quartile, the third quartile, and the interquartile range. Set the value range of the i-th column element according to the first quartile, the third quartile, and the interquartile range. Traverse the i-th column element. If there is an element that does not belong to the value range, determine that the element is an outlier. For the identified outliers, the deviation between the outlier and the range of values ​​is calculated, and it is determined whether the deviation is greater than a preset deviation threshold. If so, the outlier is analyzed and processed according to the analysis results. If not, the mean of the remaining elements in the column where the outlier is located is calculated, and the mean is used to replace the outlier.

8. A construction data analysis and processing system, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores a computer-readable storage medium, and the computer-readable storage medium stores a computer program. When the processor processes a computer program stored on the computer-readable storage medium, it implements the method as described in any one of claims 1-7.

Images