Enterprise data processing and intelligent service method based on multi-source heterogeneous data fusion

Abstract

This invention provides a method for enterprise data processing and intelligent services based on multi-source heterogeneous data fusion, belonging to the field of data processing technology. The method includes: acquiring multi-source heterogeneous data streams from the construction industry; performing anomaly detection using the Isolation Forest algorithm to identify anomalous data points; using a knowledge graph of the construction industry to perform entity semantic mapping on the anomalous data points to obtain standardized data elements; performing spatiotemporal alignment processing on the standardized data elements and fusing them with historical data to generate thematic data; constructing prediction and diagnostic models based on the core themes of progress, cost, quality, and security calculated from the fused data, using federated learning and random forest algorithms; and providing data access interfaces to the outside world through a data service gateway. This invention achieves a complete closed loop from data access to intelligent services, improving data fusion efficiency and decision support levels.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, specifically to a method for enterprise data processing and intelligent services based on the fusion of multi-source heterogeneous data. Background Technology

[0002] In the process of digital transformation of construction enterprises, data, as a core production factor, has become crucial for improving project management efficiency, optimizing resource allocation, and achieving refined operations through its effective integration and in-depth utilization. Currently, the construction industry commonly employs multiple independent business systems, including BIM design platforms, IoT monitoring systems, ERP, and supply chain management systems. This results in diverse data sources, varying structures, and inconsistent standards, creating a serious "data silo" phenomenon. The primary obstacle to addressing the data silo problem lies in the data access and preprocessing stages. Construction projects involve a wide variety of data types: BIM model data is structured geometric information, IoT sensors generate time-series data, on-site inspections collect point cloud data, business management systems output relational data, and there is also publicly available external data. These data are accessed through different transmission protocols (such as MQTT, HTTP, Modbus, etc.), resulting in vastly different formats and making direct integration and utilization difficult.

[0003] Currently, existing technologies typically employ protocol converters to access multi-source data and rely on rule engines or statistical methods for anomaly detection and cleaning. However, these methods have significant limitations: Firstly, construction site conditions are complex and variable, with diverse and often unknown types of anomalies caused by factors such as sensor malfunctions, network transmission jitter, and human input errors. Traditional rule-based or threshold-based methods, relying on expert experience and presets, struggle to cover all anomaly scenarios and lack the ability to identify unknown types of anomalies, easily leading to missed or false detections. Secondly, the same entity is often expressed using different terms in different systems; for example, "concrete strength" and "concrete strength" coexist, and "construction section" and "work surface" are used interchangeably. Traditional semantic mapping relies on manually maintaining dictionaries or simple string matching, which is not only costly to maintain and has poor scalability but also struggles to handle complex semantic relationships such as synonyms, abbreviations, and industry jargon, resulting in semantic ambiguity even in standardized data.

[0004] In view of the above, this application is hereby submitted. Summary of the Invention

[0005] This invention provides a method for enterprise data processing and intelligent services based on the fusion of multi-source heterogeneous data, which can at least partially improve the above-mentioned problems.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] A method for enterprise data processing and intelligent services based on multi-source heterogeneous data fusion, comprising: Acquire multi-source heterogeneous raw data streams in the construction field, perform unsupervised anomaly detection and quality verification processing on the multi-source heterogeneous raw data streams based on the isolated forest algorithm, and identify abnormal data points; By using a pre-built knowledge graph in the construction field to perform entity semantic mapping on abnormal data points, standardized data elements with unified semantic identifiers are obtained. The standardized data elements are spatiotemporally aligned and then fused with historical data to generate fused thematic data. The core business themes are calculated based on the fused thematic data, and a prediction and diagnostic model is constructed based on the federated learning framework and the random forest algorithm. The core business themes include schedule, cost, quality, and security. The prediction and diagnostic models are made into services, and data access interfaces are provided to the outside world through a pre-defined data service gateway.

[0008] In summary, this invention proposes a data processing and intelligent service method for the digital transformation of construction enterprises, constructing a closed-loop architecture covering the entire process from data access, anomaly detection, semantic mapping, spatiotemporal fusion, theme modeling to intelligent services. The method first accesses BIM model data, IoT time-series data, sensor point aggregates, structured data from business systems, and publicly available external data through a multi-protocol adapter. It then employs the isolated forest algorithm for unsupervised anomaly detection, identifying and marking data points with transmission and value range anomalies. Next, it utilizes a pre-constructed knowledge graph of the construction domain to perform semantic mapping on entities within the anomaly data points. Automatic alignment is achieved through vector similarity calculation; when the similarity falls below a threshold, manual confirmation is triggered, and an enhanced graph is provided. Finally, standardized data elements with unified semantic identifiers are output. Secondly, an improved ICP algorithm is used to spatially align the BIM model with sensor point clouds. Adaptive weights, Huber loss function, and regularization terms are introduced to enhance robustness. The time base of multi-source time series is unified using the least squares method to correct sampling frequency differences and clock skew. Based on this, four thematic data models—schedule, cost, quality, and safety—are constructed using a five-tuple formal definition. A batch-stream integrated processing architecture is used to fuse historical and real-time data streams to build a thematic data warehouse. Thirdly, based on the fused thematic data, four core business themes are automatically calculated. A random forest algorithm is used to construct prediction and diagnostic models for schedule prediction, cost estimation, risk level identification, and quality problem classification. When dealing with privacy data from multiple projects, a federated learning framework is introduced for privacy-preserving collaborative training, supporting gradient pruning and differential privacy enhancement. The models are encapsulated as services through standardized APIs. Finally, a unified data service gateway provides data access interfaces. Combining real-time data and prediction models, the NSGA-II multi-objective optimization algorithm is used to jointly optimize objectives such as schedule, cost, resource balance, and quality risk, generating multi-dimensional trade-off decision recommendation reports.

[0009] Compared with existing technologies, this invention has the following advantages: By deeply integrating the knowledge graph of the construction field, it enhances the ability to identify and align professional terms and industry jargon, supports manual feedback and dynamic graph enhancement, and overcomes the limitations of traditional methods that rely on text descriptions and have high maintenance costs; it proposes an improved ICP algorithm and a unified time benchmark mechanism, solving the spatial matching and time synchronization problems of BIM and sensor data, and supporting data fusion throughout the entire life cycle of construction projects; it introduces federated learning and differential privacy technology, enabling multi-project collaborative training without leaving the local data source, balancing model performance and data security, and solving privacy compliance obstacles in cross-project data collaboration; it extends from data governance to intelligent decision-making, forming a complete value chain, directly supporting business scenarios such as progress warning, cost control, quality assessment, and safety risk identification; it adopts a batch-stream integrated architecture, integrating batch processing and stream processing engines, which can perform thematic modeling on historical data and window aggregation and instant analysis on real-time data streams, meeting the dynamic management needs of construction projects; it uses the NSGA-II algorithm to jointly optimize multiple objectives such as schedule, cost, resource balance, and quality risk, generating multi-dimensional trade-off decision recommendation reports, providing project managers with quantifiable basis for solution selection. This invention systematically solves the core technical problems in the digital transformation of construction enterprises, such as low degree of multi-source data integration, weak semantic understanding ability, lack of privacy protection mechanism and incomplete decision support chain. It realizes end-to-end closed-loop processing from raw heterogeneous data to interpretable business decisions, and significantly improves data governance efficiency and intelligent decision-making level. Attached Figure Description

[0010] Figure 1 This is a flowchart illustrating the enterprise data processing and intelligent service method based on multi-source heterogeneous data fusion provided in an embodiment of the present invention. Detailed Implementation

[0011] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0012] refer to Figure 1 As shown, the first embodiment of the present invention discloses an enterprise data processing and intelligent service method based on multi-source heterogeneous data fusion, which can be executed by an enterprise data processing and intelligent service device based on multi-source heterogeneous data fusion (hereinafter referred to as data processing and intelligent service device), specifically, by one or more processors within the data processing and intelligent service device, to implement the following method: S1, acquire multi-source heterogeneous raw data streams in the construction field, perform unsupervised anomaly detection and quality verification processing on the multi-source heterogeneous raw data streams based on the isolated forest algorithm, and identify abnormal data points; Specifically, step S1 further includes: acquiring the multi-source heterogeneous raw data streams of the building domain accessed by the configurable multi-protocol adapter, and performing unsupervised anomaly detection and quality verification processing on the multi-source heterogeneous raw data streams based on the isolated forest algorithm to identify abnormal data points; By using a pre-built knowledge graph in the construction field to perform entity semantic mapping on abnormal data points, standardized data elements with unified semantic identifiers are obtained.

[0013] The multi-source heterogeneous raw data stream includes BIM model data, IoT time series data, on-site sensor point aggregation, structured data from the business management system, and externally available data.

[0014] The multi-source heterogeneous raw data stream is subjected to unsupervised anomaly detection and quality verification processing based on the Isolation Forest algorithm. Specifically, the multi-source heterogeneous raw data stream is used as a dataset. and set This represents the number of trees in the Isolation Forest algorithm. For subsampling size, The maximum depth is given by n, where n is the number of samples and m is the number of features; where, the isolated forest is composed of... It consists of isolated trees (iTree); The construction process for each isolated tree is as follows: from the dataset Random selection Each sample is used as a training subset of the current tree. When the number of samples at the current node is determined... Or all sample feature values ​​are the same or the tree depth reaches 100%. When the split stops, the current node becomes a leaf node; When the number of samples in the current node has not been determined Or all sample feature values ​​are the same or the tree depth reaches 100%. At that time, a feature q and a splitting value p are randomly selected, where the splitting value p is between the minimum and maximum values ​​of the current node sample on feature q; When judged When a sample is assigned to the left subtree and the rest to the right subtree, repeat the above steps until all leaf nodes are generated, thus obtaining the forest. , where x is a sample The first isolated tree, For the first An isolated tree; Standardized function , , For harmonic numbers, Let Euler's constant be 1. For harmonic numbers The number of terms; Calculate the average path length for each sample , Let x be the path length of sample x in the i-th tree; When it is determined that the path ends at a leaf node and that node contains multiple samples, the path length is adjusted using the following formula: , This represents the number of samples contained in the leaf node. For the standardized function, as defined above The same applies; it is used to correct path length deviations caused by a greater than 1 sample in a leaf node. Calculate anomaly scores The formula is: and abnormal scores With preset threshold Perform a comparison, and when it is determined When a sample x is identified as an abnormal data point, the abnormal data points include transmission abnormal data points and value range abnormal data points.

[0015] S2, through a pre-built knowledge graph of the construction field, performs entity semantic mapping processing on abnormal data points to obtain standardized data elements with unified semantic identifiers; Specifically, step S2 further includes: obtaining a pre-built knowledge graph of the architecture domain, wherein the knowledge graph... By triplet Its composition, its formula is: , For a collection of entities, Let h be a set of relations, t be the head entity in the triple, t be the tail entity in the triple, and r be the relation connecting the head entity and the tail entity. For each entity and relationships To randomly initialize a d-dimensional vector, the formula is: , For the vectorized representation of entity e, The vector space formed by vectorizing all entities. Let r be the vectorized representation of relation r. Let be the d-dimensional vector space formed by vectorizing all relations; Calculate the triplet for each positive sample. distance ,in, The vector of the head entity h, Let be the vector of the tail entity t. It is an L2 norm; Randomly replace the head entity of the triple or tail entity , Generate negative samples and calculate the negative sample distance of the triplet that replaces the head entity. Or replace the negative sample distance of the triplet of the tail entity , The vector of the new head entity h' after replacement. The vector of the new tail entity t' after replacement; Calculate the interval loss of the triplet of the replacement head entity. Or the interval loss of the triplet replacing the tail entity , For interval hyperparameters that are greater than 0; The gradient descent method is used to update the vector, and its formula is as follows: , , The learning rate; For the entities in the aforementioned abnormal data points Use the preset BERT model to obtain vectors , For the BERT model; For each standard entity in the knowledge graph Calculate cosine similarity Find the most similar standard entities based on cosine similarity. , Given the i-th dimension coordinate value of the input entity vector, The coordinate value of the i-th dimension of the standard entity vector; when When, automatic mapping is performed to obtain standardized data elements with unified semantic identifiers; when When manual confirmation is triggered, standardized data elements with unified semantic identifiers are obtained, among which, The input entity is the cosine similarity to the most similar standard entity. This is the similarity threshold for knowledge graph mapping, used to determine whether automatic mapping can be performed.

[0016] In this embodiment, based on adaptive protocol parsing and semantic mapping, multi-source data access and standardization are implemented. Preliminary data quality checks are performed on the original data stream to identify and mark data points with transmission anomalies and value range anomalies. Semantic standardization is then performed on the data that passes the checks: a pre-constructed knowledge graph of the construction domain is built to map standard business entities into low-dimensional vectors; for entities in the input data, the similarity between their vector representations and candidate entity vectors in the knowledge graph is calculated; when the highest similarity exceeds a preset threshold, the mapping is automatically completed, transforming heterogeneous original data from different sources into standardized data elements with unified semantic identifiers and assigning them unified standard codes; otherwise, a manual confirmation process is triggered, and the confirmation result is fed back to the knowledge graph to enhance its mapping capabilities.

[0017] This step employs entity alignment technology based on knowledge graphs and vector similarity. By pre-constructing a knowledge graph for the construction industry, business entities are vectorized, and automatic alignment and encoding allocation are achieved through similarity calculation. It supports manual feedback and dynamic graph enhancement, thereby improving the accuracy of terminology mapping.

[0018] S3, perform spatiotemporal alignment processing on the standardized data elements, and fuse them with historical data to generate fused thematic data; Specifically, step S3 further includes: performing spatial alignment processing on the standardized data elements, aligning the BIM model and sensor spatial coordinates using the ICP algorithm, and aligning multi-source time series using a time base unification method to achieve time alignment processing and obtain a real-time data stream; Based on four themes—schedule, cost, quality, and safety—a thematic data model is constructed. Based on the thematic data model, a batch-stream integrated processing architecture is used to fuse historical data and real-time data streams to generate fused thematic data. The fused thematic data is written into the corresponding data table of the thematic data warehouse of the thematic data model.

[0019] The standardized data elements are subjected to spatiotemporal alignment processing, specifically: for BIM model data in multi-source heterogeneous raw data streams. and on-site sensor point aggregation Key points are extracted and initial correspondences are established using preset feature descriptors (such as FPFH and SHOT). ,in, For the i-th point in the BIM model, This represents the total number of points in the BIM model. Let j be the j-th point in the sensor point cloud. This represents the total number of points in the sensor point cloud. For the k-th pair of correspondences, the point in the BIM model, For the points in the sensor point cloud set in the k-th pair of correspondences, This represents the initial total number of corresponding point pairs. For the set of real numbers, Represents a point in three-dimensional space; Establish the optimization formula: To solve for the optimal rigid body transformation , Let be a rotation matrix. It is a translation vector. For adaptive weights, For point-to-point distance, The Huber loss function is used to enhance the algorithm's robustness to outliers. The regularization coefficient is . The regularization term ensures the orthogonality of the rotation matrices, and K is the total number of corresponding point pairs used in the current iteration or after filtering. This is a simplified form of optimal rigid body transformation; where, , The Huber loss threshold; The optimization is performed by alternating between singular value decomposition (SVD) and gradient descent, specifically by sequentially calculating the weighted centroids. Centralized point set Weighted covariance matrix SVD decomposition of the weighted covariance matrix Calculate the rotation matrix ,make sure And calculate the translation vector. ,in, For transpose, For the weighted centroid of the BIM point set, The weighted centroid of the sensor point set, For the centralized point set of the BIM model, For the centralized point set of the sensor, This is the left singular vector matrix obtained by performing SVD decomposition on the covariance matrix H. Let V be the transpose of matrix V. This is the right singular vector matrix obtained by performing SVD decomposition on the covariance matrix H. This is the transpose of matrix U. Given the determinant of the matrix, ensure that the rotation matrix... The determinant is +1; Rematching point pairs until the convergence condition is met. Or reach the maximum number of iterations The final transformation matrix is ​​obtained. This completes the spatial alignment of standardized data elements, among which... The nearest neighbor search algorithm is used to find new corresponding point pairs between the transformed BIM point cloud and the sensor point cloud. The matrix after the i-th iteration transformation Transformed BIM point cloud, Let be the rotation matrix obtained in the i-th iteration. Let be the rotation matrix obtained in the (i-1)th iteration. Let Frobenius norm be the matrix. Let be the translation vector obtained in the i-th iteration. Let be the translation vector obtained in the (i-1)th iteration. This is the convergence threshold, used to determine whether the iteration has stopped. The final optimized rotation matrix is... This is the final optimized translation vector; Use linear transformation to convert timestamps from different systems To unify it into a standard time base, the formula is: , This is the time scaling factor (to correct for differences in sampling frequencies). This is the time offset (correcting for clock skew). This is the random error term; Using the least squares method on a given reference time series With source time series The estimation is performed to complete the time alignment of standardized data elements, and the formula is as follows: , , Where N is the total number of timestamp pairs used to estimate the time alignment parameter. This is the i-th reference timestamp (standard time). For the i-th source timestamp (the device time that needs to be aligned). The mean of the reference time series, This is the mean of the source time series.

[0020] Based on a thematic data model, a batch-stream integrated processing architecture is used to fuse historical data and real-time data streams to generate fused thematic data. Specifically, based on the thematic data model, a batch-stream integrated processing architecture is used to fuse historical data using a preset batch processing engine, and a stream processing engine is used to perform window aggregation on the real-time data stream to generate fused thematic data. The thematic data model is formally defined using a 5-tuple, and its mathematical expression is: , This is a collection of entities, representing the core objects in the business domain. A set of attributes that describes the characteristics and states of an entity. For a set of relations, define the semantic relationships between entities. To constrain the set and ensure data compliance for business purposes, This is a set of computational functions used to perform derived computations on business themes, wherein the theme data model includes a schedule theme model. Cost Theme Model Quality Topic Model Security Theme Model .

[0021] In this embodiment, data fusion and thematic database construction (i.e., designing thematic data models oriented towards core building business) are based on spatiotemporal consistency constraints and thematic modeling. Standardized data is fused according to business themes to build a thematic data warehouse that supports historical traceability and real-time analysis. This step employs an improved ICP algorithm to achieve spatial alignment between BIM and sensor data, unify the time base, and construct four thematic data models for schedule, cost, quality, and safety, supporting batch and stream processing, thus realizing multi-source data fusion and thematic modeling under spatiotemporal consistency constraints.

[0022] The design of the four core theme models is as follows: (1) Progress Topic Model The entity is defined as: ={Project, Task, Milestone, Resource}; Key attribute examples: ={Task ID, Task Name, Planned Start Time, Planned End Time, Actual Start Time, Actual End Time, Percentage Completed, Critical Path Identifier}; Relationship Definition: Task Dependencies: ,in ={FS,SS,FF,SF} is a set of dependency types, where FS, SS, FF, and SF represent finish-start, start-start, finish-finish, and start-finish, respectively; business constraints include: time sequence constraints: Schedule logic constraint: If task t i FS (Complete-Start) depends on task t j Thematic data model, then t i Actual start time t j Actual completion time; Derivative topic calculation function: Schedule deviation rate: Total float time on the critical path: LF, EF, LS, and ES are the latest completion time, earliest completion time, latest start time, and earliest start time, respectively.

[0023] (2) Cost Theme Model Entity definition: ={Cost Item, Budget Item, Actual Expenditures, Change Orders}; Key Attributes: Cost Item Entity Attributes: ={Cost Item ID, Budget Amount, Actual Amount, Change Amount, Overspending Rate}; Relationship Definition: Cost Allocation Relationship: The third component represents the allocation ratio; Business constraints: Budget constraints: Cash flow constraints: ; Derivative topic calculation function: Cost-performance index: Cost deviation: = Budgeted cost of completed work The actual cost of the work already completed.

[0024] (3) Quality Theme Model Entity definition: ={Checkpoint, Quality Standard, Measured Data, Rectification Items}; Key Attributes: Checkpoint Entity Attributes: ={Checkpoint ID, Inspection Standard, Measured Value, Allowable Deviation, Pass / Fail Status}; Relationship Definition: Inspection Attribution Relationship: The third component represents the importance weight. Business constraints: Quality standard constraints: Rectification Time Limit: Rectification items must be closed within the specified time. Derivative Topic Calculation Function: Quality Pass Rate: Quality risk index: ,in As weight, The severity level. (4) Security Theme Model Entity definition: ={Risk Events, Safety Hazards, Safety Measures, Emergency Response Plans}; Key Attributes: Risk Event Entity Attributes: ={Risk ID, Risk Type, Probability of Occurrence, Impact Level, Risk Grade}; Relationship Definition: Risk Trigger Relationship: The third component represents the triggering condition logic expression. Business constraints: Risk acceptable constraints: Where the risk value = probability of occurrence Impact level; Safety measure implementation constraints: High-risk tasks must be equipped with corresponding safety measures; Derivative topic calculation function: Safety risk index: ,in Let the probability of occurrence be... Accident frequency statistics to reflect the degree of impact: .

[0025] S4. Calculate core business themes based on fused topic data, and construct prediction and diagnostic models based on federated learning framework and random forest algorithm. The core business themes include schedule, cost, quality, and security. Specifically, step S4 further includes: calculating four core business themes—progress, cost, quality, and security—based on the integrated theme data; A collaborative modeling environment is built based on the federated learning framework. In this environment, the random forest algorithm is used to perform privacy-preserving collaborative training on multiple project data to obtain prediction and diagnostic models. In particular, when privacy data of multiple projects is involved, privacy-preserving collaborative training is performed based on the federated learning framework. The prediction formula for the regression tasks (schedule prediction, cost estimation) of the prediction and diagnostic model is as follows: The prediction formula for classification tasks (risk level, quality problem classification) is: Where B is the number of trees in the random forest, and c is the class label in the classification task. Let b be the prediction result of the b-th decision tree for sample x. This is an indicator function. It evaluates to 1 if the condition within the parentheses is true, and 0 otherwise.

[0026] In this embodiment, business topics and intelligent models are constructed based on random forest and federated learning. Based on the fused data, key business topics are automatically calculated and reusable prediction and diagnostic models are trained. Specifically, this step integrates the intelligent modeling framework of random forest and federated learning. A random forest prediction model is constructed based on topic data, and a federated learning framework is introduced to achieve collaborative training with multi-project privacy protection, supporting gradient pruning and differential privacy enhancement.

[0027] Specifically, the construction of a building intelligence model based on random forest includes the following steps: (1) Determine the basic algorithm of random forest, which is an ensemble learning model composed of multiple decision trees. (2) The specific training steps of random forest in the field of architecture are as follows: First, data preparation and feature engineering, the feature set of the architecture field: Feature importance selection: Calculation of feature importance using the built-in features of Random Forest. ,in, Let N be the number of samples at node t, and N be the total number of samples. The amount of impurity reduction for splitting node t using feature j.

[0028] Secondly, self-sampling, for each tree T b Sample from the original data D Sampling with replacement; expected proportion of unsampled data: .

[0029] Secondly, random feature selection: at each node split, m features are randomly selected from p features: classification task: Return to task: .

[0030] Finally, in the decision tree construction algorithm, node impurity is measured by Gini impurity (for classification tasks). ,in The proportion of class c in node t. Mean squared error (regression task): ,in Let be the mean value of node t. Split gain calculation: Where s is the split point, , The left and right child nodes, Impurity of node t. Stopping condition: Number of samples n at node t. t n min All characteristics are the same, and the amount of impurity is reduced. , reaching the maximum depth d max .

[0031] Specifically, in this embodiment, the federated learning training steps are as follows: First, the initialization phase, the central server initializes the global model: ,in, This is the initial model parameter vector. To initialize the variance, σ is the standard deviation of the Gaussian distribution, and I is the identity matrix, representing that each dimension is independent and identically distributed. d represents the model parameter dimension, i.e., the number of parameters in the neural network. Client selection strategy: m clients are selected in each round of communication. ,in Let be the set of clients selected in the t-th round of communication. , Select the scale for the client. The total number of clients participating in federated learning.

[0032] The second step, the local training phase, involves client k receiving the global model: , Here are the local model parameters for client k at the start of the t-th round of communication, with the superscript (t, 0) indicating the 0th local iteration in the t-th round; the local loss function (taking security risk prediction as an example): ,in, Let k be the number of samples locally owned by client k. (Cross-entropy loss) This represents the model's predicted value for the input x. The sigmoid activation function maps the linear transformation result to the (0,1) interval. The feature extraction function is used. Local gradient descent update: for e = 1, 2, ..., E (local iterations): , ,in, Let g represent the gradient calculated by client k in the t-th round of communication and the e-th local iteration. Let Fk be the gradient operator of the loss function. This is the small batch of data for round e. Here, E represents the learning rate, and E represents the number of local training epochs. Gradient clipping (for privacy protection): ,in, Here, C represents the clipped gradient vector, and C is the clipping threshold, which controls the upper bound of the gradient norm. Let L2 be the norm of the gradient vector.

[0033] Step 3, model upload, local model update count: Differential privacy noise addition: Noise standard deviation: ,in, To budget for privacy, control the level of privacy protection. For privacy failure probability, Let k be the model update amount for client k in the t-th round of communication. This represents the model update amount after adding differential privacy noise. It is Gaussian noise with a mean of 0 and a covariance matrix of... .

[0034] Step 4, safe aggregation phase, weighted average aggregation: ,in, This represents the total number of samples from participating clients in this round. Weighted by data volume.

[0035] Step 5, Model Distribution and Validation, Global Model Distribution: All clients, local verification: Each client is evaluated on its local verification set. ,in, Let k be the accuracy of the client on the local validation set. Let k be the number of samples in the local validation set of the client. As an indicator function, when the predicted value Equal to reality It takes the value 1 if it is true, otherwise it takes the value 0.

[0036] S5 provides a service-oriented architecture for the prediction and diagnostic models, and offers data access interfaces to external users through a pre-defined data service gateway. Specifically, it provides online prediction and batch inference services by offering standardized APIs to the trained prediction and diagnostic models.

[0037] Specifically, this also includes: combining service-oriented prediction and diagnostic models with real-time data flow, and using multi-objective optimization algorithms to generate decision-making recommendation reports tailored to business scenarios. The multi-objective optimization algorithm employs a multi-objective optimization decision system based on NSGA-II. It includes decision variables and an objective function, where the decision variables are: ,in, Let be the task start time, r be the resource allocation matrix, c be the cost allocation vector, and q be the quality input vector. Four optimization objectives: (Construction period) (cost), (Resource balance) (Quality risks), among which, For the duration of task i, For resource j, the unit cost Let t represent the resource usage at time j. For active task collection, Average resource usage For standard quality input, This represents the weight of task importance.

[0038] In this embodiment, scenario-based intelligent services are generated based on multi-objective optimization algorithms. Utilizing the data, topics, and models generated in the preceding steps, decision support services tailored to specific business scenarios are produced. Specifically, a gateway integrates a query optimizer, enabling the decomposition of complex business queries into parallel subqueries. Combining real-time data and predictive models, a multi-objective optimization algorithm is applied to generate decision recommendation reports containing multiple trade-offs. This method combines real-time data and predictive models, employing the NSGA-II multi-objective optimization algorithm to generate decision-making schemes that balance multiple objectives such as schedule, cost, resources, and quality, providing scientific decision support. Furthermore, a full-process system architecture is designed, encompassing data access, semantic standardization, spatiotemporal fusion, intelligent modeling, and decision services, to achieve closed-loop support for the digital transformation of construction enterprises.

[0039] The specific steps of the NSGA-II core algorithm are as follows: (1) Population initialization: Randomly generate the initial population. Each individual is randomly generated: ,in, For population size, , For the boundary of the decision variables, (2) Evaluate fitness: Calculate the four objective function values ​​F1, F2, F3, and F4 for each individual, as well as the constraint violation. : ,in, For the k-th inequality constraint, it is required that ≤0, For the l-th equality constraint, it is required that =0, For when When >0, take The value is 0 if the condition is not met. for The absolute value of. (3) Non-dominated solution sorting: Individual Dominate (recorded as) ), if and only if: , (4) Calculate the crowding distance for each front edge F. i The individual in F i The i-th Pareto front, i.e., the set of individuals with the same non-dominated level, sorted by each objective function value, is the boundary individual: ,in, This refers to the first individual (boundary individual) in the current frontier after sorting by target m. For the last individual (boundary individual) in the current frontier after sorting by target m, the intermediate individuals are: , For individuals Crowding distance, a measure of the density around an individual, where, , Let m be the maximum and minimum values ​​of the target m in the current frontier. (5) Randomly select two individuals. , Comparison rule: Prioritize the smaller rank. choose When ranks are the same, choose the one with higher crowding: choose (6) Simulate binary crossover, parent generation: For each dimension i=1,...,D: , Offspring: , ,in, The distribution index controls the range of crossovers. The random numbers are uniformly distributed. Let be the crossover coefficient in the i-th dimension, controlling the similarity between offspring and parents. Let be the values ​​of the two offspring individuals in the i-th dimension. (7) Polynomial variation, for individuals Let i be each dimension: , ,in, , After mutation: ,in, Let be a random number in the i-th dimension, used to determine the direction of mutation. Let be the coefficient of variation of the i-th dimension. Let xi be the relative distance from the lower limit. Let xi be the relative distance from the upper limit. Let i be the value of the i-th dimension after the mutation. The variation distribution index. (8) Environmental selection, merging parent generation Pt and offspring generation Qt: Perform non-dominated sorting on Rt, and fill the new population Pt+1 according to the frontier hierarchy. When a frontier cannot be fully filled, select individuals from the crowding distance in descending order. The selection strategy is as follows: ,in, These are the k individuals with the largest crowding distance in the Lth front. , according to Sort in descending order.

[0040] Specifically, in this embodiment, the main steps of the decision support method are: constructing a decision matrix. Normalization processing is performed. (3) Determine the ideal solution and the negative ideal solution , (4) Calculate the distance , (5) Calculate the relative proximity The larger the value, the better the solution.

[0041] In summary, at the data access level, this method employs a configurable multi-protocol adapter, compatible with various data sources such as BIM models, IoT time-series data, structured data from business management systems, and publicly available external data. It introduces an unsupervised anomaly detection model based on isolated forests for real-time quality verification of the data stream. Furthermore, through a pre-built knowledge graph of the construction domain and vector similarity calculation, it achieves automatic alignment and standardized encoding of entity semantics, supporting manual feedback and dynamic graph enhancement, significantly improving the accuracy and adaptability of industry terminology mapping. At the data fusion level, a multi-source data alignment mechanism under spatiotemporal consistency constraints is proposed: an improved ICP algorithm is used for spatial registration of BIM models and on-site sensor point clouds, and a unified benchmark correction for multi-source time series is achieved based on the least squares method. On this basis, four core business theme data models—progress, cost, quality, and safety—are designed, constructing a batch-stream integrated processing architecture that supports batch fusion of historical data and real-time data stream window aggregation, forming a thematic data warehouse with both historical traceability and real-time response capabilities.

[0042] At the intelligent modeling level, four core business themes are automatically calculated based on the fused thematic data, and machine learning models for scenarios such as schedule prediction, cost estimation, and risk identification are constructed using the random forest algorithm. For the group-level multi-project collaborative modeling needs, a federated learning framework and differential privacy mechanism are introduced to achieve global model collaborative training without data leaving the local machine, effectively balancing model performance and data security. At the decision service level, a unified data service gateway provides standardized data and model service interfaces. Combining real-time perceived data and predictive model outputs, the NSGA-II multi-objective optimization algorithm is used to jointly optimize objectives such as schedule, cost, resource balance, and quality risk, generating multi-dimensional trade-off decision-making solutions for business personnel to reference, realizing the value extension from data governance to intelligent decision-making. In summary, this method, through deep integration of industry knowledge, spatiotemporal alignment technology, privacy computation mechanism, and multi-objective optimization methods, constructs a complete data processing and intelligent service solution adapted to the digital transformation scenarios of construction enterprises, achieving significant improvements in improving the efficiency of multi-source heterogeneous data fusion, enhancing business modeling capabilities, ensuring data collaboration security, and supporting scientific decision-making.

[0043] Compared with existing technologies, this method has the following advantages: 1. Deep integration of industry knowledge enhances the professionalism of semantic understanding. By introducing a knowledge graph in the construction field, it enhances the ability to identify and align professional terms and industry jargon, overcoming the limitations of traditional methods that rely on text descriptions. 2. Achieves spatiotemporal alignment, adapting to the fusion of multi-source data from construction sites. An improved ICP algorithm and time unification mechanism are proposed, solving the spatial matching and time synchronization problems of BIM and sensor data, supporting data fusion throughout the entire lifecycle of construction projects. 3. Integrates privacy-preserving computation mechanisms, supporting secure collaborative modeling. Federated learning and differential privacy technologies are introduced to enable multi-project collaborative training without centralized data, balancing data security and model performance. 4. Extends from data governance to intelligent decision-making, forming a complete value chain. It not only achieves data standardization and fusion but also further constructs predictive models and optimizes decision-making services, directly supporting business scenarios such as progress warnings, cost control, and quality assessment. 5. Adopts a batch-stream integrated architecture, supporting real-time analysis and historical traceability. It integrates batch processing and stream processing engines, enabling both thematic modeling of historical data and window aggregation and real-time analysis of real-time data streams, thus meeting the dynamic management needs of construction projects.

[0044] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A method for enterprise data processing and intelligent services based on multi-source heterogeneous data fusion, characterized in that, include: Acquire multi-source heterogeneous raw data streams in the construction field, perform unsupervised anomaly detection and quality verification processing on the multi-source heterogeneous raw data streams based on the isolated forest algorithm, and identify abnormal data points; By using a pre-built knowledge graph in the construction field to perform entity semantic mapping on abnormal data points, standardized data elements with unified semantic identifiers are obtained. The standardized data elements are spatiotemporally aligned and then fused with historical data to generate fused thematic data. The core business themes are calculated based on the fused thematic data, and a prediction and diagnostic model is constructed based on the federated learning framework and the random forest algorithm. The core business themes include schedule, cost, quality, and security. The prediction and diagnostic models are made into services, and data access interfaces are provided to the outside world through a pre-set data service gateway.

2. The enterprise data processing and intelligent service method based on multi-source heterogeneous data fusion according to claim 1, characterized in that, The multi-source heterogeneous raw data stream includes BIM model data, IoT time series data, on-site sensor point aggregation, structured data from the business management system, and publicly available external data.

3. The enterprise data processing and intelligent service method based on multi-source heterogeneous data fusion according to claim 1, characterized in that, The multi-source heterogeneous raw data stream is subjected to unsupervised anomaly detection and quality verification processing based on the isolated forest algorithm to identify anomalous data points, specifically as follows: The multi-source heterogeneous raw data stream is used as the dataset. and set This represents the number of trees in the Isolation Forest algorithm. For subsampling size, The maximum depth is denoted by , where n is the number of samples and m is the number of features; The steps for constructing each isolated tree are as follows: from the dataset Random selection Each sample is used as a training subset of the current tree. When the number of samples at the current node is determined... Or all sample feature values ​​are the same or the tree depth reaches When the split stops, the current node becomes a leaf node; Recursively execute the following steps until the termination condition is met: when the number of samples for the current node has not been determined. Or all sample feature values ​​are the same or the tree depth reaches At that time, a feature q and a splitting value p are randomly selected, where the splitting value p is between the minimum and maximum values ​​of the current node sample on feature q; When judged When a sample is assigned to the left subtree and the rest to the right subtree, repeat the above steps until all leaf nodes are generated, thus obtaining the forest. , where x is a sample The first isolated tree, For the first An isolated tree; Define the normalization function , , For harmonic numbers, Let Euler's constant be 1. For harmonic numbers The number of terms; Calculate the average path length for each sample , Let x be the path length of sample x in the i-th tree; When it is determined that the path ends at a leaf node and that node contains multiple samples, the path length is adjusted using the following formula: , This represents the number of samples contained in the leaf node. Calculate anomaly scores The formula is: and abnormal scores With preset threshold Perform a comparison, and when it is determined When a sample x is identified as an abnormal data point, the abnormal data points include transmission abnormal data points and value range abnormal data points.

4. The enterprise data processing and intelligent service method based on multi-source heterogeneous data fusion according to claim 3, characterized in that, By using a pre-built knowledge graph of the construction domain to perform entity semantic mapping on abnormal data points, standardized data elements with unified semantic identifiers are obtained, specifically: Obtain a pre-built knowledge graph for the architecture domain, where the knowledge graph... By triplet Its composition, its formula is: , For a collection of entities, Let h be a set of relations, t be the head entity in the triple, t be the tail entity in the triple, and r be the relation connecting the head entity and the tail entity. For each entity and relationships To randomly initialize a d-dimensional vector, the formula is: , For the vectorized representation of entity e, The vector space formed by vectorizing all entities. Let r be the vectorized representation of relation r. Let be the d-dimensional vector space formed by vectorizing all relations; Calculate the triplet for each positive sample. distance ,in, The vector of the head entity h, Let be the vector of the tail entity t. It is an L2 norm; Randomly replace the head entity of the triple or tail entity , Generate negative samples and calculate the negative sample distance of the triplet that replaces the head entity. Or replace the negative sample distance of the triplet of the tail entity , The vector of the new head entity h' after replacement. The vector of the new tail entity t' after replacement; Calculate the interval loss of the triplet of the replacement head entity. Or the interval loss of the triplet replacing the tail entity , For interval hyperparameters that are greater than 0; The gradient descent method is used to update the vector, and its formula is as follows: , , The learning rate; For the entities in the aforementioned abnormal data points Use the preset BERT model to obtain vectors , For the BERT model; For each standard entity in the knowledge graph Calculate cosine similarity Find the most similar standard entities based on cosine similarity. ; The L2 norm is: , Given the i-th dimension coordinate value of the input entity vector, The coordinate value of the i-th dimension of the standard entity vector; when When, automatic mapping is performed to obtain standardized data elements with unified semantic identifiers; when When manual confirmation is triggered, standardized data elements with unified semantic identifiers are obtained, among which, The input entity is the cosine similarity to the most similar standard entity. This is the similarity threshold for knowledge graph mapping.

5. The enterprise data processing and intelligent service method based on multi-source heterogeneous data fusion according to claim 4, characterized in that, The standardized data elements are subjected to spatiotemporal alignment processing and fused with historical data to generate fused thematic data, specifically as follows: The standardized data elements are spatially aligned by using the ICP algorithm to align the BIM model with the sensor spatial coordinates and by using a time base unification method to align the multi-source time series, thereby achieving time alignment and obtaining a real-time data stream. Based on four themes—schedule, cost, quality, and safety—a thematic data model is constructed. Based on the thematic data model, a batch-stream integrated processing architecture is used to fuse historical data and real-time data streams to generate fused thematic data. The fused thematic data is written into the corresponding data table of the thematic data warehouse of the thematic data model.

6. The enterprise data processing and intelligent service method based on multi-source heterogeneous data fusion according to claim 5, characterized in that, The standardized data elements are subjected to spatiotemporal alignment processing, specifically as follows: BIM model data in multi-source heterogeneous raw data streams and on-site sensor point aggregation Key points are extracted and initial correspondences are established using preset feature descriptors. ,in, For the i-th point in the BIM model, This represents the total number of points in the BIM model. Let j be the j-th point in the sensor point cloud. This represents the total number of points in the sensor point cloud. For the k-th pair of correspondences, the point in the BIM model, For the points in the sensor point cloud set in the k-th pair of correspondences, This represents the initial total number of corresponding point pairs. For the set of real numbers, Represents a point in three-dimensional space; Establish the optimization formula: To solve for the optimal rigid body transformation , Let be a rotation matrix. It is a translation vector. For adaptive weights, For point-to-point distance, Here is the Huber loss function. The regularization coefficient is . Here, K is the regularization term, and K is the total number of corresponding point pairs used in the current iteration or after filtering. This is a simplified form of optimal rigid body transformation. The optimization employs alternating singular value decomposition and gradient descent, specifically: calculating the weighted centroids sequentially. Centralized point set Weighted covariance matrix SVD decomposition of the weighted covariance matrix Calculate the rotation matrix ,make sure And calculate the translation vector. ,in, For transpose, For the weighted centroid of the BIM point set, The weighted centroid of the sensor point set, For the centralized point set of the BIM model, For the centralized point set of the sensor, This is the left singular vector matrix obtained by performing SVD decomposition on the covariance matrix H. Let V be the transpose of matrix V. This is the right singular vector matrix obtained by performing SVD decomposition on the covariance matrix H. This is the transpose of matrix U. Let be the determinant of the matrix; Rematching point pairs until the convergence condition is met. Or reach the maximum number of iterations The final transformation matrix is ​​obtained. This completes the spatial alignment of standardized data elements, among which... For nearest neighbor search algorithm, The BIM point cloud is the result of the i-th iteration transformation matrix. Let be the rotation matrix obtained in the i-th iteration. Let be the rotation matrix obtained in the (i-1)th iteration. Let Frobenius norm be the matrix. Let be the translation vector obtained in the i-th iteration. Let be the translation vector obtained in the (i-1)th iteration. The convergence threshold, The final optimized rotation matrix is... This is the final optimized translation vector; Use linear transformation to convert timestamps from different systems To unify it into a standard time base, the formula is: , This is the time scaling factor. This is the time offset. This is the random error term; Using the least squares method on a given reference time series With source time series The estimation is performed to complete the time alignment of standardized data elements, and the formula is as follows: , , Where N is the total number of timestamp pairs used to estimate the time alignment parameter. For the i-th reference timestamp, For the i-th source timestamp, The mean of the reference time series, This is the mean of the source time series.

7. The enterprise data processing and intelligent service method based on multi-source heterogeneous data fusion according to claim 6, characterized in that, Based on a thematic data model, a batch-stream integrated processing architecture is used to fuse historical data and real-time data streams to generate fused thematic data. Specifically, based on the thematic data model, a batch-stream integrated processing architecture is used to fuse historical data using a preset batch processing engine, and a stream processing engine is used to perform window aggregation on the real-time data stream to generate fused thematic data. The thematic data model is formally defined using a 5-tuple, and its mathematical expression is: , For a collection of entities, For a set of relations, For a collection of attributes, For a constraint set, The set of computational functions, wherein the topic data model includes a progress topic model. Cost Theme Model Quality Topic Model Security Theme Model .

8. The enterprise data processing and intelligent service method based on multi-source heterogeneous data fusion according to claim 7, characterized in that, Based on the fused thematic data, core business themes are calculated, and prediction and diagnostic models are constructed using a federated learning framework and random forest algorithm, specifically as follows: Based on the integrated thematic data, four core business themes are calculated: progress, cost, quality, and safety. A collaborative modeling environment is built based on the federated learning framework. In this environment, the random forest algorithm is used to perform privacy-preserving collaborative training on multiple project data to obtain prediction and diagnostic models. In particular, when privacy data of multiple projects is involved, privacy-preserving collaborative training is performed based on the federated learning framework. The regression prediction formula for the prediction and diagnosis model is as follows: The formula for predicting classification tasks is: Where B is the number of trees in the random forest, and c is the class label in the classification task. Let b be the prediction result of the decision tree for sample x. This is an indicator function.

9. The enterprise data processing and intelligent service method based on multi-source heterogeneous data fusion according to claim 1, characterized in that, Also includes: By combining the service-oriented prediction and diagnostic models with real-time data flow, a multi-objective optimization algorithm is used to generate decision-making recommendation reports for business scenarios. The multi-objective optimization algorithm adopts a multi-objective optimization decision system based on NSGA-II.

Images