A method and system for cross-project transfer of engineering equipment knowledge

Abstract

This invention provides a method and system for cross-project knowledge transfer of engineering equipment, belonging to the field of engineering project technology. The method includes: using an improved temporal block algorithm to divide source project data into multiple subdomains with the most diverse distributions; using a hybrid deep neural network model to extract hidden knowledge for each subdomain, thereby extracting the knowledge to be transferred from the source project data; and using a multi-knowledge feature fusion method to fuse the extracted hidden knowledge from each subdomain to obtain target project data, thus completing the cross-project transfer of engineering equipment knowledge. This invention solves the problems of existing knowledge transfer methods that lack consideration for removing noise from source domain data due to differences in data distribution, and do not simultaneously utilize multiple types of knowledge to make the model more adaptable to the target domain data when fusion results are obtained.

Description

Technical Field

[0001] This invention belongs to the field of engineering project technology, and in particular relates to a method and system for cross-project transfer of engineering equipment knowledge. Background Technology

[0002] Project knowledge is a crucial resource for project-based organizations, and improving project performance through managing project knowledge is a hot topic in project management. As a key aspect of project knowledge management, the effective extraction and transfer of project knowledge has become a critical research issue. The extraction and transfer of knowledge related to engineering equipment (such as tunnel boring machines) is a core component of project knowledge and a major research focus. By fully extracting and transferring the knowledge contained in engineering equipment across projects, timely support for decision-making by equipment management and operators can be provided, effectively mitigating decision-making errors caused by a lack of knowledge and experience in the early stages of a project. However, due to the differences in geological conditions, equipment parameters, and the nature of the construction project itself, cross-project knowledge transfer is difficult to utilize effectively. Therefore, enhancing the cross-project adaptability of engineering equipment knowledge, and thus improving its efficiency and effectiveness, is a key and challenging issue that project knowledge management urgently needs to address. Currently, research on the extraction and transfer of engineering equipment knowledge includes the following aspects:

[0003] (1) Based on theoretical models and numerical analysis. Focusing on the interaction between the shield and the soil, the stress mechanism during the shield tunneling process is studied by modeling a large number of geological parameters.

[0004] (2) Data-driven. Utilizing the large amount of construction data generated during the tunnel boring process, machine learning models and deep learning models are established to learn the complex mapping relationship between construction data and key tunneling parameters.

[0005] (3) Based on knowledge fusion. Historical construction data is divided into multiple data blocks according to a specific strategy, and engineering equipment knowledge is extracted and integrated from each data block.

[0006] Existing research shows that the performance of theoretical models and numerical analysis methods is limited by theoretical assumptions and parameter settings. Machine learning-based methods struggle to handle high-dimensional, nonlinear time-series data and cannot capture high-dimensional time-series information. While deep learning-based methods have made leaps in model expressive power, they are limited by the sample size of training data, making it difficult to meet the accuracy requirements of target project prediction tasks. To address the problems of sparse training data and inapplicable knowledge to be transferred, and to improve the utilization rate of knowledge from existing construction projects, transfer learning has emerged. Transfer learning allows the use of historical construction data from the source project to complete the cross-project transfer of knowledge from the source domain to the target domain, improving the accuracy of target project prediction tasks. However, directly using all available construction data from the source project for knowledge extraction and transfer may present some problems. First, equipment sensor data usually arrives in the form of streaming data, which causes the data distribution of the source project itself to change dynamically over time. The data distribution of the source project contains noise, which also introduces noise into the knowledge extracted by using all the data from the source project. Second, due to differences in project environment and operating conditions, the data distribution of the source project and the target project will have distance differences. Even if a learning model is developed that highly fits the data distribution of the source project, it is difficult to apply it to a target project with a different data distribution. Therefore, consideration should be given to reducing noise in the cross-project transfer of engineering equipment knowledge, fully extracting available engineering equipment knowledge, and enhancing the adaptability of cross-project knowledge to the target project.

[0007] A review of existing technical literature revealed that current techniques do not consider removing noise from source domain data caused by differences in data distribution, nor do they simultaneously utilize multiple types of knowledge to make the model more adaptable to the target domain data when fusing the results. Therefore, there is an urgent need to design a method that can reduce negative transfer caused by differences in data distribution while improving the adaptability of the knowledge to be transferred to the target domain. Summary of the Invention

[0008] To address the aforementioned shortcomings in existing technologies, this invention provides a method and system for cross-project transfer of engineering equipment knowledge. This method solves the problems of existing knowledge transfer methods, such as the lack of consideration for removing source domain data, noise caused by differences in data distribution, and the failure to simultaneously utilize multiple types of knowledge to make the model more adaptable to the target domain data when fusing results. This invention effectively realizes cross-project transfer of engineering equipment knowledge.

[0009] To achieve the above objectives, the technical solution adopted by this invention is: a method for cross-project transfer of engineering equipment knowledge, comprising the following steps:

[0010] S1. Using an improved time-series block algorithm, the source project data is divided into multiple subdomains with the most diverse distribution.

[0011] S2. Utilize a hybrid deep neural network model to extract hidden knowledge for each subdomain in order to extract knowledge to be transferred from the source project data;

[0012] S3. Utilize multi-knowledge feature fusion methods to fuse the extracted hidden knowledge from each subdomain to obtain target project data and complete the cross-project transfer of engineering equipment knowledge.

[0013] The beneficial effects of this invention are as follows: To reduce noise in the cross-project transfer of engineering equipment knowledge, fully extract available engineering equipment knowledge, and enhance the adaptability of cross-project knowledge to the target project by utilizing multiple types of knowledge, this invention uses completed projects (or projects with a large amount of accumulated data) as source project data. It learns the mapping relationship between potential influencing factors and predictive variables from their construction data, which is the transferable project knowledge. This knowledge is then used for the prediction task of projects under construction (target projects), accurately predicting the shield machine attitude and position deviation at a certain time interval in the future. This solves the technical problems of existing knowledge transfer methods that do not consider removing source domain data, noise caused by differences in data distribution, and do not simultaneously utilize multiple types of knowledge to make the model more adaptable to the target domain data when fusing results. This effectively realizes the cross-project transfer of engineering equipment knowledge.

[0014] Further, step S1 includes the following steps:

[0015] S101. Based on the sample size of the source project data and prior knowledge, determine the search set N for the number of candidate split points and the search set K for the number of split points.

[0016] S102. Traverse each element in the search set N to determine n candidate split points, wherein the n candidate split points divide the source project data into n+1 equal blocks;

[0017] S103. Traverse each element in the search set K to obtain the key split point p. k Get the current distribution difference d k ;

[0018] S104. Based on the maximum entropy theory, return the maximum distribution difference d from step S103. k Let k* be the number of corresponding split points, and let k* corresponding split points be denoted as k*. in, This represents k* split points;

[0019] S105. Backtrack the value of n in step S102 to obtain the maximum distribution difference d. k The corresponding value of n is denoted as n*, where n represents the number of candidate split points;

[0020] S106. Based on the processing results of steps S105 to S106, obtain the multiple subdomains E with the most diverse distributions:

[0021] E = k* + 1.

[0022] The beneficial effects of the above-mentioned further scheme are: by considering the time-varying characteristics of the source project data distribution, the source project data is divided into multiple subdomains with the most diverse distribution. This satisfies the consistency of data distribution within each subdomain while maximizing the difference in data distribution between each subdomain, thereby eliminating data noise caused by the time-varying distribution of the source project and facilitating subsequent multi-knowledge fusion using the domain-specific knowledge and distribution knowledge of the subdomains.

[0023] Furthermore, step S2 includes the following steps:

[0024] S201. For the e-th subdomain, use a convolutional neural network (CNN) to extract knowledge containing spatial representation information, where e∈E;

[0025] S202. Using the Long Short-Term Memory (LSTM) network, knowledge containing spatial representation information is integrated to obtain hidden temporal information.

[0026] S203. Use the skip connections of residual networks to address the degradation problem of hybrid deep neural network models;

[0027] S204. Using a fully connected layer, the obtained hidden temporal information is fused, the mapping relationship between feature representation and truth value is learned, and the hidden knowledge of each subdomain is extracted to realize the extraction of knowledge to be transferred from the source project data.

[0028] The beneficial effects of the above-mentioned further solution are: while capturing the time and space dimensions of the source project, the hybrid deep neural network model can avoid network degradation, thereby enabling the effective extraction of rich knowledge from the source project and facilitating knowledge transfer and reuse.

[0029] Furthermore, step S3 includes the following steps:

[0030] S301. Based on a hybrid deep neural network model, by acquiring several subdomain knowledge extraction models, the regression loss of the e-th subdomain knowledge extraction model is obtained.

[0031] S302. Freeze the feature extractor parameters of the subdomain knowledge extraction model and transfer them to the target domain. The feature extractor based on the e-th subdomain knowledge extraction model extracts the e-th group of deep feature representations of the target domain respectively to obtain the distribution difference knowledge of the high-level feature representations between each subdomain and the target domain.

[0032] S303. Based on the regression loss and distribution difference knowledge, a multi-knowledge fusion feature extractor is obtained, and the training data of the target domain is input to obtain the high-level feature representation after multi-knowledge feature fusion.

[0033] S304. After the feature extractor of multi-knowledge fusion, a fully connected layer is spliced ​​to serve as the regressor for the target project prediction task. Based on the feature extractor and regressor of multi-knowledge fusion, a prediction model for the target project is constructed.

[0034] S305. Based on the high-level feature representation after the fusion of multiple knowledge features, the prediction model of the target project is fine-tuned, and the target project data is obtained by using the fine-tuned prediction model of the target project, thus completing the cross-project transfer of engineering equipment knowledge.

[0035] The beneficial effects of the above-mentioned further scheme are as follows: Firstly, domain-specific knowledge encompasses the learning ability of each subdomain knowledge extraction model for its subdomain, i.e., the explanatory power of the subdomain knowledge extraction model for parameter variables and predictor variables. When a subdomain knowledge extraction model can fit the predictor variables on the source project well, the domain-specific knowledge extracted by the subdomain knowledge extraction model should be transferred. Secondly, domain distribution knowledge reflects the distribution of data within a subdomain, based on which the distribution difference values ​​between multiple pairs of subdomains and the target project can be obtained. When the distribution difference of a domain pair (a subdomain of the source domain and the target domain constitute a domain pair) is small, it indicates that the knowledge extracted from the subdomain is better suited to the task of the target project. In summary, by simultaneously utilizing the domain-specific knowledge of each subdomain and the distribution knowledge of the subdomains and the target domain, and employing knowledge fusion weights to fuse multiple subdomain knowledge extraction models, the fused model can be made more adaptable to the target project.

[0036] Furthermore, the expression for the regression loss of the knowledge extraction model for the e-th subdomain is as follows:

[0037] dist_reg e =∑|p e -y e |

[0038] p e =RS e (fsx e ;θ RSe )

[0039]

[0040] Among them, dist_reg e p represents the regression loss of the knowledge extraction model for the e-th subdomain. e y represents the output value of the regressor of the knowledge extraction model for the e-th subdomain. e RS represents the true value of the e-th subdomain. e() represents the regression layer of the knowledge extraction model for the e-th subdomain, fsx e θ represents the deep feature representation of the e-th subdomain extracted by the feature extractor of the knowledge extraction model for the e-th subdomain. RSe Let FEe() represent the training parameters of the regression layer of the knowledge extraction model for the e-th subdomain, and let sx represent the feature extractor of the subdomain knowledge extraction model. e This represents the input value of the e-th subdomain of the knowledge extraction model for the e-th subdomain. represents the training parameters of the feature extractor of the knowledge extraction model for the e-th subdomain.

[0041] The beneficial effects of the above-mentioned further solutions are: for construction data with high data dimensionality and strong temporal sequence, by adopting the constructed knowledge extraction model, the spatial and temporal information hidden in the data can be fully mined, thereby realizing the full extraction of knowledge, which is conducive to the transfer and reuse of knowledge in subsequent processes.

[0042] Furthermore, the expression for the distribution difference knowledge of the high-level feature representations of each subdomain and the target domain is as follows:

[0043] dist_d e =MMd(fsx) e' ftx e )

[0044]

[0045] Among them, dist_d e MMd() represents the distributional difference knowledge between the high-level feature representations of the e-th subdomain and the target domain extracted by the feature extractor of the e-th subdomain knowledge extraction model, where MMd() represents the MMD distance metric, and fsx e' ftx represents the feature representation of the e-th subdomain extracted by the knowledge extraction model of the e-th subdomain. e tx represents the feature representation of the target domain extracted by the knowledge extraction model of the e-th subdomain. e This represents the input value of the target domain for the knowledge extraction model of the e-th subdomain. represents the training parameters of the feature extractor of the knowledge extraction model for the e-th subdomain.

[0046] The beneficial effect of the above-mentioned further scheme is that by using MMD distance to measure the distance between the feature representations of the e-th subdomain and the target domain, the difference in their distance distribution can be reflected, thereby providing key guidance for multi-knowledge feature fusion in subsequent steps.

[0047] Furthermore, the expression for the high-level feature representation after the fusion of multiple knowledge features is as follows:

[0048] ftx=Add(ftx'1,ftx'2,...,ftx' E ;θ add )

[0049] ftx E =Multiply(w e ftx e ;θ Mul )

[0050]

[0051] Where ftx represents the high-level feature representation after the fusion of multiple knowledge features, Add() represents the Add neural network layer in deep learning, which has the function of vector summation, ftx' E Indicates w e and FTX e The output after processing by the multiply layer, θ Mul and θ add These represent the training parameters for the Multiply and Add layers, respectively. Multiply() represents the Multiply neural network layer in deep learning, which functions as a matrix multiplication layer. e ftx represents the fusion weights of the feature extractors in the knowledge extraction model for the e-th subdomain. e This represents the feature representation of the target domain extracted by the knowledge extraction model of the e-th subdomain, and MinMaxScaler() represents the normalization operation on the selected content.

[0052] The beneficial effects of the above-mentioned further approach are: domain-specific knowledge reflects the explanatory power of the subdomain knowledge extraction model for parameter variables and predictor variables, while domain distribution knowledge reflects the distributional differences between the subdomain and the target item. Based on the domain-specific knowledge of each subdomain and the distributional knowledge of the subdomain and the target domain, the resulting fusion model will achieve better results on the target item.

[0053] Furthermore, step S305 specifically includes:

[0054] Based on the high-level feature representation obtained after the fusion of multiple knowledge features, the prediction model for the target project is fine-tuned using the following formula. Then, using the fine-tuned prediction model, the target project data is obtained, completing the cross-project transfer of engineering equipment knowledge:

[0055] p t =RT(ftx; θ) RT )

[0056] Where, p t θ represents the output value of the target project prediction model, RT() represents the regressor used for the target project prediction task, and θRT This represents the training parameters of the regression layer in a knowledge extraction model that integrates multiple knowledge sources.

[0057] The beneficial effect of the above-mentioned further scheme is that by adopting a fine-tuning method based on the prediction model of the target project, not only will the general knowledge obtained from multiple subdomains be retained, but also the specific knowledge from the target domain will be added, thereby making the fusion prediction model more adaptable to the target project and showing good generalization performance on the target task.

[0058] This invention provides a cross-project knowledge transfer system for engineering equipment, comprising:

[0059] The source project time series segmentation module is used to divide the source project data into multiple subdomains with the most diverse distribution using an improved time series segmentation algorithm.

[0060] The source project knowledge extraction module is used to extract hidden knowledge in each subdomain using a hybrid deep neural network model, so as to extract the knowledge to be transferred from the source project data.

[0061] The feature fusion module based on multi-knowledge is used to fuse the extracted hidden knowledge of each subdomain using multi-knowledge feature fusion methods to obtain target project data and complete the cross-project transfer of engineering equipment knowledge.

[0062] The beneficial effects of this invention are as follows: To reduce noise in the cross-project transfer of engineering equipment knowledge, fully extract available engineering equipment knowledge, and enhance the adaptability of cross-project knowledge to the target project by utilizing multiple types of knowledge, this invention uses completed projects (or projects with a large amount of accumulated data) as source project data. It learns the mapping relationship between potential influencing factors and predictive variables from their construction data, which is the transferable project knowledge. This knowledge is then used for the prediction task of projects under construction (target projects), accurately predicting the shield machine attitude and position deviation at a certain time interval in the future. This solves the technical problems of existing knowledge transfer methods that do not consider removing source domain data, noise caused by differences in data distribution, and do not simultaneously utilize multiple types of knowledge to make the model more adaptable to the target domain data when fusing results. This effectively realizes the cross-project transfer of engineering equipment knowledge. Attached Figure Description

[0063] Figure 1 This is a flowchart of the method of the present invention.

[0064] Figure 2 This invention relates to the feature fusion method MKF based on multiple knowledge.

[0065] Figure 3 This is a graph showing the prediction results of the TBMKF framework of the present invention on the target project.

[0066] Figure 4This is a schematic diagram of the system structure of the present invention. Detailed Implementation

[0067] The specific embodiments of the present invention are described below to enable those skilled in the art to understand the present invention. However, it should be understood that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, various changes are obvious as long as they are within the spirit and scope of the present invention as defined and determined by the appended claims. All inventions utilizing the concept of the present invention are protected.

[0068] Example 1

[0069] This invention eliminates data noise caused by time-varying distribution in the source project data by applying an improved time-series block algorithm, resulting in multiple subdomains. Using a constructed hybrid deep neural network model, it extracts the hidden transferable knowledge from these subdomains. By implementing a designed multi-knowledge-based feature fusion method, it integrates the transferable knowledge from multiple subdomains, making it more suitable for the target project. Ultimately, this assists in the target project's prediction tasks and improves its management performance. Figure 1 As shown, this invention provides a method for cross-project transfer of engineering equipment knowledge, the implementation of which is as follows:

[0070] S1. Using an improved time-series block partitioning algorithm, the source project data is divided into multiple subdomains with the greatest distribution diversity. The implementation method is as follows:

[0071] S101. Based on the sample size of the source project data and prior knowledge, determine the search set N for the number of candidate split points and the search set K for the number of split points.

[0072] S102. Traverse each element in the search set N to determine n candidate split points, wherein the n candidate split points divide the source project data into n+1 equal blocks;

[0073] S103. Traverse each element in the search set K to obtain the key split point p. k Get the current distribution difference d k ;

[0074] S104. Based on the maximum entropy theory, return the maximum distribution difference d from step S103. k Let k* be the number of corresponding split points, and let k* corresponding split points be denoted as k*. in, This represents k* split points;

[0075] S105. Backtrack the value of n in step S102 to obtain the maximum distribution difference d. k The corresponding value of n is denoted as n*, where n represents the number of candidate split points;

[0076] S106. Based on the processing results of steps S105 to S106, obtain the multiple subdomains E with the most diverse distributions.

[0077] In this embodiment, an improved temporal block partitioning algorithm (TBME) is applied to the source project data to divide the source project data into multiple subdomains with the greatest distribution diversity, specifically as follows:

[0078] Based on the sample size of the source domain data and prior knowledge in the case, determine the search set N for the number of candidate split points and the search set K for the number of split points; traverse each element n in the search set N to determine n candidate split points, which can divide the source domain data into n+1 equal blocks; traverse each element k in the search set K (in the case, k is exhaustively searched, let k=n). When k=1, starting from the first candidate split point and going up to the kth split point, obtain the sum of the distribution differences of the subdomains obtained if that point is split, and return the key split point p1 that maximizes the sum of distribution differences. At this time, the optimal split point set is {p1}, and record the current distribution difference d1; When k=2, execute the greedy strategy, fix the already obtained key split point p1, and from the remaining k-1 candidate split points, return the key split point p2 that maximizes the sum of distribution differences according to the same selection strategy. At this time, the optimal split point set is {p1,p2}, and record the current distribution difference d2; ...; When k=k, obtain the key split point p k At this point, the optimal set of split points is {p1, p2, ..., p...} k Record the current distribution difference d. k Based on the maximum entropy theory, return the maximum distribution difference d from step S103. k Let k* be the number of corresponding split points, and let k* corresponding split points be denoted as k*. Backtracking the value of n in step S102, we obtain the maximum distribution difference d. k The corresponding value of n is denoted as n*. Thus, we have obtained the multiple subdomains E with the greatest distributional diversity:

[0079] E = k* + 1.

[0080] S2. Using a hybrid deep neural network model, hidden knowledge in each subdomain is extracted to extract knowledge to be transferred from the source project data. The implementation method is as follows:

[0081] S201. For the e-th subdomain, use a convolutional neural network (CNN) to extract knowledge containing spatial representation information, where e∈E;

[0082] S202. Using the Long Short-Term Memory (LSTM) network, knowledge containing spatial representation information is integrated to obtain hidden temporal information.

[0083] S203. Use the skip connections of residual networks to address the degradation problem of hybrid deep neural network models;

[0084] S204. Using a fully connected layer, the obtained hidden temporal information is fused, the mapping relationship between feature representation and truth value is learned, and the hidden knowledge of each subdomain is extracted to realize the extraction of knowledge to be transferred from the source project data.

[0085] In this embodiment, the constructed hybrid deep neural network model (Res_CL) is used to extract the hidden knowledge of each subdomain, specifically as follows:

[0086] For the e-th subdomain (e∈E), firstly, a convolutional neural network (CNN) is used to eliminate data noise caused by the nonlinear dependence of sensor parameters and extract knowledge containing spatial representation information. Next, a long short-term memory (LSTM) layer is used to integrate the knowledge extracted by the CNN and further capture the hidden temporal information. Then, the skip links of the residual network are used to solve the network degradation problem. Finally, a fully connected layer is used to integrate the feature representations obtained by the feature extractor and learn the mapping relationship between the feature representations and the ground truth.

[0087] Among them, the subdomain knowledge extraction model Res_CL is as follows: Figure 2 As shown in the figure, Feature extractor; Regressor; Short connection; Input data; True value; FP; BP; C+ index: indicates the convolutional neural network layer; L+ index: indicates the long short-term memory network layer; F+ index: indicates the fully connected layer. The calculation method is detailed below:

[0088] For knowledge extraction from subdomains, this invention utilizes Convolutional Neural Networks (CNNs) to filter data noise and extract spatial feature representations at different scales. A CNN is a parameter-sharing, locally connected feedforward neural network, typically composed of an input layer, convolutional layers, activation layers, pooling layers, fully connected layers, and an output layer. The CNN extracts features through convolutional and pooling layers, inputs them into one or more fully connected layers, and finally outputs the regression results through the output layer. Specifically, the convolutional layer uses convolution kernels to convolve the output of the previous network layer and constructs the output of the current network layer using a non-linear activation function, as shown in the formula:

[0089] x k=f(w k x k-1 +b k (1)

[0090] Where k represents the (k-th)th layer of the convolutional neural network, w represents the weights, b represents the bias, f() represents the activation function, and x k-1 x represents the output of the (k-1)th layer of the convolutional neural network. k This represents the output of the k-th layer of the convolutional neural network.

[0091] Nonlinear activation functions typically use the rectified linear unit (ReLU), which can cause some neurons to output zero, improving network sparsity and mitigating overfitting. The formula for calculating ReLU is:

[0092] a k =f(y k )=max{0,y k} (2)

[0093] Among them, y k a represents the output value of the convolution operation. k Indicates y k The activation value.

[0094] Next, a Long Short-Term Memory (LSTM) network is used to capture the long-term dependencies of the time series. LSTM is one of the most widely used RNN models, avoiding gradient vanishing or inflation by introducing multiple thresholds. The storage units of an LSTM network include three threshold structures: a forget gate, an input gate, and an output gate. This threshold structure simulates switching control of information input and output, allowing training errors to propagate forward and backward, thus achieving model convergence. The gates in an LSTM network include an input gate, a forget gate, and an output gate, calculated using the following formula:

[0095] f t =σ(w f ·[h t-1 ,x t ]+b f (3)

[0096] i t =σ(w i ·[h t-1 ,x t ]+b i (4)

[0097]

[0098] o t=σ(w o ·[h t-1 ,x t ]+b o (7)

[0099] h t =o t ·tanh(c t (8)

[0100] d t =σ(w d ·h t +b d (9)

[0101] Among them, f t This represents the output obtained after passing through the forgetting threshold, σ() represents the activation function, and w f The weight h represents the forget threshold. t-1 x represents the output at the previous time step. t b represents the original input at the current moment. f Represents the bias vector, i t w represents the output obtained after passing through the input threshold. i The weight h represents the input threshold. t-1 b represents the output at the previous time step. i This represents the bias vector of the input threshold. w represents the temporary state of the cell at the current moment. c b represents the weights of the layers in a Tanh neural network. c c represents the bias vector of a Tanh neural network layer. t-1 w represents the cell state at the previous time step. o The weight b represents the output threshold. o c represents the bias vector of the output threshold. t d represents the unit state vector. t w represents the output at the current time step after passing through the Dense neural network layer. d b represents the weights of a Dense neural network layer. d Represents the bias vector of a Dense neural network layer, [h t-1 ,x t The symbol ] represents concatenating two vectors into a longer vector.

[0102] First, formula (3) constitutes the forget gate, which is determined by setting the connection matrix w. f To determine the output vector h from the previous time step t-1 Which information to remove or retain?

[0103] Secondly, equations (4), (5), and (6) constitute the input gate, which determines the unit state vector c. t How should it be updated?

[0104] Then, equations (7) and (8) form the output gate, which is used to obtain the latest unit state vector c. t Then, the output vector h is obtained by using the tanh() activation function. t Equation (9) uses the Dense layer to obtain the feature representation of the output.

[0105] P = Dense(Add[x,d]) (10)

[0106] Next, Equation (10) uses the Add layer to link the spatial feature representation x extracted by Equation (1) and the spatial feature representation d containing temporal information captured by Equations (3)-(9), so that the features of the shallow layer can also be passed to the deep layer, ensuring the performance of the deep network. Then, by using the fully connected layer Dense to construct the regressor of the subdomain knowledge extraction model, the mapping relationship between the feature representation and the ground truth is learned. In the equation, P represents the predicted value of the subdomain knowledge extraction model.

[0107] S3. Utilizing a multi-knowledge feature fusion method, the extracted hidden knowledge from each subdomain is fused to obtain the target project data, completing the cross-project transfer of engineering equipment knowledge. The implementation method is as follows:

[0108] S301. Based on a hybrid deep neural network model, by acquiring several subdomain knowledge extraction models, the regression loss of the e-th subdomain knowledge extraction model is obtained.

[0109] S302. Freeze the feature extractor parameters of the subdomain knowledge extraction model and transfer them to the target domain. The feature extractor based on the e-th subdomain knowledge extraction model extracts the e-th group of deep feature representations of the target domain respectively to obtain the distribution difference knowledge of the high-level feature representations between each subdomain and the target domain.

[0110] S303. Based on the regression loss and distribution difference knowledge, a multi-knowledge fusion feature extractor is obtained, and the training data of the target domain is input to obtain the high-level feature representation after multi-knowledge feature fusion.

[0111] S304. After the feature extractor of multi-knowledge fusion, a fully connected layer is spliced ​​to serve as the regressor for the target project prediction task. Based on the feature extractor and regressor of multi-knowledge fusion, a prediction model for the target project is constructed.

[0112] S305. Based on the high-level feature representation after the fusion of multiple knowledge features, the prediction model of the target project is fine-tuned, and the target project data is obtained by using the fine-tuned prediction model of the target project, thus completing the cross-project transfer of engineering equipment knowledge.

[0113] In this embodiment, the structure of the hybrid deep neural network model is fixed. This invention utilizes this fixed-structure model to train the dataset of each subdomain (assuming there are E subdomains), thus training a total of E models and obtaining the regression loss of each E model. These E models not only include the fixed model structure itself, but also contain model parameters capable of fitting each subdomain dataset.

[0114] In this embodiment, the subdomain knowledge extraction model includes a feature extractor and a regressor. In this invention, the feature extractor comprises a stack of Convolutional Neural Networks (CNN), Long Short-Term Memory Networks (LSTM), and residual networks, while the regressor is a stack of fully connected layers.

[0115] Therefore, step S2, which involves extracting hidden knowledge from various subdomains using a hybrid deep neural network model, actually comprises two smaller steps. Step ① involves the feature extractor extracting feature representations from the original data, and step ② involves the regressor learning the mapping relationship between the feature representations and the true values.

[0116] In this embodiment, the feature fusion method MKF based on multi-knowledge is as follows: Figure 3 As shown in the figure, `subdomain + index samples` indicates the input data for the specified subdomain; `target domain samples` indicates the input data for the target domain; `dist_d` indicates the input data for the specified subdomain. e : The distribution difference loss of the high-level feature representations of the e-th subdomain and the target domain extracted by the e-th subdomain feature extractor; dist_reg e : Regression loss of the e-th knowledge extraction model; ftx: fused feature representation; Training parameters of the feature extractor of the knowledge extraction model in the e-th subdomain; FE + subscript: the feature extractor of the knowledge extraction model in the nth subdomain; RS + subscript: the regressor of the knowledge extraction model in the nth subdomain; RT: the regressor of the target domain; pretrained model; finetuned model; forward propagation; backward propagation / gradient update; parameter transfer; knowledge source transfer. The calculation method is detailed below:

[0117] First, a knowledge extraction model Res_CL is pre-trained using source domain data. Based on step S2, E = k*+1 subdomain knowledge extraction models containing domain-specific and distributional knowledge from different subdomains can be obtained. The feature extractor FE is then used based on the e-th (e∈[1,E]) subdomain knowledge extraction model. e (), extract the deep feature representation of the e-th subdomain, as shown in Equation (11). The regressor based on the knowledge extraction model of the e-th subdomain maps the deep feature representation to the regression truth value, as shown in Equation (12). Furthermore, the regression loss of the model of the e-th subdomain is calculated as one of the knowledge bases for feature fusion, as shown in Equation (13).

[0118]

[0119] p e =RS e (fsx e ;θ RSe (12)

[0120] dist_reg e =∑|p e -y e | (13)

[0121] Among them, dist_reg e p represents the regression loss of the knowledge extraction model for the e-th subdomain. e y represents the output value of the regressor of the knowledge extraction model for the e-th subdomain. e RS represents the true value of the e-th subdomain. e () represents the regression layer (including fully connected layers F1, F2, and F3) of the knowledge extraction model for the e-th subdomain, fsx e θ represents the deep feature representation of the e-th subdomain extracted by the feature extractor of the knowledge extraction model for the e-th subdomain. RSe Let FEe() represent the training parameters of the regression layer of the knowledge extraction model for the e-th subdomain, and let sx represent the feature extractor of the subdomain knowledge extraction model. e This represents the input value of the e-th subdomain of the knowledge extraction model for the e-th subdomain. represents the training parameters of the feature extractor of the knowledge extraction model for the e-th subdomain.

[0122] Secondly, the parameters of the feature extractor of the subdomain knowledge extraction model are frozen and transferred to the target domain. Based on the feature extractor of the e-th subdomain knowledge extraction model, the e-th group of deep feature representations of the target domain are extracted, as shown in Equation (14). Furthermore, the distribution difference knowledge of the high-level feature representations of each subdomain and the target domain can be obtained as one of the knowledge bases for feature fusion, as shown in Equation (15).

[0123]

[0124] dist_d e =MMd(fsx) e' ftx e (15)

[0125] Among them, dist_d e MMd() represents the distributional difference knowledge between the high-level feature representations of the e-th subdomain and the target domain extracted by the feature extractor of the e-th subdomain knowledge extraction model, where MMd() represents the MMD distance metric, and fsx e' ftx represents the feature representation of the e-th subdomain extracted by the knowledge extraction model of the e-th subdomain. e tx represents the feature representation of the target domain extracted by the knowledge extraction model of the e-th subdomain. e This represents the input value of the target domain for the knowledge extraction model of the e-th subdomain. represents the training parameters of the feature extractor of the knowledge extraction model for the e-th subdomain.

[0126] Then, using the training data of the target domain, feature fusion of multiple knowledge is performed. Based on the feature extractor of the pre-trained E subdomain knowledge extraction model, the knowledge fusion weights are calculated using the knowledge obtained from equations (13) and (15), resulting in the knowledge fusion feature extractor MKF_FE(). Inputting the training data of the target domain, the high-level feature representation ftx after multi-knowledge feature fusion is obtained, as shown in the following equation:

[0127]

[0128] ftx E =Multiply(w e ftx e ;θ Mul (17)

[0129] ftx=Add(ftx'1,ftx'2,...,ftx' E ;θ add (18)

[0130] Where ftx represents the high-level feature representation after the fusion of multiple knowledge features, Add() represents the Add neural network layer in deep learning, which has the function of vector summation, ftx' E Indicates w e and FTX e The output after processing by the multiply layer, θ Mul and θ addThese represent the training parameters for the Multiply and Add layers, respectively. Multiply() represents the Multiply neural network layer in deep learning, which functions as a matrix multiplication layer. e ftx represents the fusion weights of the feature extractors in the knowledge extraction model for the e-th subdomain. e This represents the feature representation of the target domain extracted by the knowledge extraction model of the e-th subdomain, and MinMaxScaler() represents the normalization operation on the selected content.

[0131] Finally, based on the constructed multi-knowledge fusion feature extractor, a fully connected layer is spliced ​​onto the back of its network to serve as the regressor RT() for the target project prediction task, as shown in Equation (19). The multi-knowledge fusion feature extractor MKF_FE() and the regressor RT() constitute the target project prediction model. By optimizing Equation (20), the regression layer of this model is fine-tuned to make the prediction model more adaptable to the target project prediction task.

[0132] p t =RT(ftx; θ) RT (19)

[0133]

[0134] Where RT() represents the regressor (including fully connected layers F1, F2, and F3) as the target item prediction task, θ RT p represents the training parameters of the regression layer in a knowledge extraction model based on multi-knowledge fusion. t This represents the output value of the target project prediction model.

[0135] The following explanation will take the implementation of the temporal block multi-knowledge fusion knowledge transfer framework TBMKF proposed in this invention during shield tunneling as an example.

[0136] I. Data Processing

[0137] (1) Constructing case data

[0138] This invention utilizes a large amount of data recorded during tunnel boring machine (TBM) construction (including data from numerous sensors on systems such as guidance, propulsion, cutterhead, and screw conveyor) to learn multi-timestep knowledge for TBM attitude prediction and achieve cross-project knowledge transfer across different geological environments and TBM equipment. This invention collects real data from four construction projects (S, O, P, and Q) using different geological environments and TBM equipment as experimental datasets. Based on the differences in construction data sample size, working conditions, and equipment, project S, with richer data, is selected as the source project. Project S shares the same construction equipment and geological conditions as project O, but differs from P and Q. O, P, and Q are then used as target projects, constructing six transfer task scenarios, each with a different transfer scenario.

[0139] Each construction dataset has 70 input variables. The variables to be predicted are a set of shield head deviation variables recorded by the guidance system, namely shield head horizontal deviation (HDSH) and shield head vertical deviation (VDSH). In addition, prediction experiments were conducted for each shield deviation variable for five time spans (t+1, t+2, t+3, t+4, t+5), that is, the model prediction time points were set to one time period, two time periods, three time periods, four time periods, and five time periods backward, respectively.

[0140] (2) Data preprocessing

[0141] (a) Null value handling. In the null value handling stage, all rows with empty column values ​​are directly deleted. For rows that are not all empty but have empty column values, the column values ​​are filled with 0.

[0142] (b) Duplicate value handling. In the duplicate data processing stage, for duplicate row data, only the first occurrence of the row data is retained, and subsequent duplicate row data are deleted.

[0143] (c) Outlier handling. During the outlier handling phase, based on construction experience, rows with ring numbers less than -10 are deleted; rows with cutterhead rotation speed greater than 0 are selected from rings 0-50; for other intermediate rings, rows with tunneling speed greater than 0 and cutterhead rotation speed greater than 0.5 are selected.

[0144] (d) Non-tunneling data processing. Delete construction data in non-tunneling and shutdown states.

[0145] (e) Data normalization. The experimental data were normalized using the MinMaxScaler method, scaling the data to the range [0,1].

[0146] II. Method Implementation

[0147] (1) In the time-series partitioning stage, by solving Algorithm 1, the number of optimal candidate split points N*=9 and the number of optimal split points K*=1 are returned.

[0148] (2) In the knowledge extraction stage, the hidden knowledge of each subdomain is extracted using the knowledge extraction model Res_CL constructed in this invention, utilizing the training samples of each subdomain. In the pre-training stage of the knowledge extraction model, pre-training models need to be established for subdomains with different sample sizes. Random search yields iteration counts n of 100 and 150, a learning rate μ of 1e-2, and batch sizes of 128 and 256. During the pre-training process of the engineering equipment knowledge extraction model for the source project, the amount of training data is large, so RMSProp is selected as the optimizer for model training.

[0149] (3) In the multi-knowledge-based feature fusion stage, the knowledge fusion method MKF designed in this invention is used to design feature fusion weights by utilizing the domain-specific knowledge of multiple subdomains and the distribution difference knowledge between the subdomains and the target project, resulting in a multi-knowledge fusion feature extractor MKF_FE. Based on the multi-knowledge fusion feature extractor MKF_FE, the knowledge to be transferred from the source project can be extracted. The knowledge to be transferred is then transferred to the target project to assist in the prediction task on the target project. In the fine-tuning stage, a random search yields an iteration count n of 80, a learning rate μ of 1e-2, and a batch size of 64. In the algorithm fine-tuning stage, to make the pre-trained model more suitable for the target project, Adam is selected as the optimizer for model fine-tuning.

[0150] (4) In the model prediction and indicator evaluation stages, this invention uses R 2 The performance of TBMKF is evaluated using R-Square, Mean Absolute Error (MAE), and Root Mean Square Error (RMSE), and the calculation formulas are as follows:

[0151]

[0152] Among them, y i p represents the true value corresponding to the i-th test sample. i This represents the predicted value of the proposed method for the i-th test sample. The average value represents the true values, and n represents the total number of test samples.

[0153] The prediction results of the TBMKF method proposed in this invention on the target project are shown in the attached figure. Figure 3 .

[0154] In summary, by adopting the above technical solution, the advantages of this invention are:

[0155] 1. Unlike existing deep learning-based knowledge extraction models, the hybrid deep neural network model Res_CL constructed in this invention is applicable to high-dimensional, time-series-based construction data, and also has universality for industrial data with similar characteristics. It not only uses Convolutional Neural Network (CNN) layers to eliminate data noise caused by the nonlinear dependencies of sensor parameters, but also integrates the knowledge extracted by CNN layers using Long Short-Term Memory (LSTM) layers to further capture the hidden temporal information, and uses skip links in residual networks to address network degradation. By constructing this knowledge extraction model, it is possible to fully and effectively extract engineering equipment knowledge contained in high-dimensional, time-series-based engineering construction data, laying a solid foundation for subsequent knowledge transfer and reuse.

[0156] 2. Unlike existing prediction models based on simple ensembles, this invention considers data distribution differences and reduces noise caused by the time-varying distribution of source item data by implementing the Time-Series Blocking (TBME) strategy. By performing time-series blocking on the source item data, multiple subdomains with consistent distribution within subdomains and maximized distribution differences between subdomains are obtained. Furthermore, the constructed subdomain knowledge extraction model is used to extract domain-specific knowledge and distribution difference knowledge in each subdomain.

[0157] 3. Unlike existing prediction models based on simple fusion, this invention enhances the adaptability of knowledge learned from completed projects for use in target projects. Based on domain-specific knowledge and knowledge of distributional differences, knowledge fusion weights are designed to fuse knowledge extraction models from multiple subdomains of completed projects, making the prediction model more adaptable to the prediction task of the target project.

[0158] Example 2

[0159] like Figure 4 As shown, the present invention provides a cross-project knowledge transfer system for engineering equipment as described in Embodiment 1, comprising:

[0160] The source project time series segmentation module is used to divide the source project data into multiple subdomains with the most diverse distribution using an improved time series segmentation algorithm.

[0161] The source project knowledge extraction module is used to extract hidden knowledge in each subdomain using a hybrid deep neural network model, so as to extract the knowledge to be transferred from the source project data.

[0162] The feature fusion module based on multi-knowledge is used to fuse the extracted hidden knowledge of each subdomain using multi-knowledge feature fusion methods to obtain target project data and complete the cross-project transfer of engineering equipment knowledge.

[0163] In this embodiment, to reduce noise in the source project data distribution, fully extract available engineering equipment knowledge, and enhance the adaptability of cross-project knowledge to the target project by utilizing multiple types of knowledge, thereby enabling early prediction of the attitude and position deviation during shield tunneling in the target project, this invention proposes a cross-project engineering equipment knowledge transfer system (TBMKF) based on temporal block multi-knowledge fusion. It mainly includes a source project temporal block module, a source project knowledge extraction module, and a feature fusion module based on multi-knowledge.

[0164] The source project time-series segmentation module is used to divide the source project into multiple subdomains with consistent data distribution within the domain and the greatest differences in data distribution between domains. This eliminates domain distribution noise in the source project and ensures the domain specificity of each subdomain and the diversity of data distribution across multiple subdomains.

[0165] The source project knowledge extraction module extracts knowledge to be transferred from source projects with a large amount of historical construction data by constructing a hybrid deep neural network model and storing it in the network model.

[0166] Based on the feature fusion module of multi-knowledge, the knowledge fusion weight is designed by utilizing the domain-specific knowledge of each subdomain and the distribution knowledge of the subdomain and the target domain. The knowledge extraction models of multiple subdomains are fused together, and the resulting prediction model will be more adapted to the target project.

[0167] This invention solves the technical problems of existing knowledge transfer methods that do not consider removing source domain data, noise caused by differences in data distribution, and do not simultaneously utilize multiple types of knowledge to make the model more adaptable to the target domain data when fusion results, thus effectively realizing cross-project transfer of engineering equipment knowledge.

Claims

1. An engineered equipment knowledge cross-project transfer method, characterized by, The method comprises the following steps: S1. dividing source project data into a plurality of sub-domains with the most distribution diversity by using an improved time-series blocking algorithm; The step S1 comprises the following steps: S101. determining a search set N of candidate split point numbers and a search set K of split point numbers in combination with a sample size of the source project data and prior knowledge; S102. determining n candidate split points by traversing each element in the search set N, wherein the n candidate split points divide the source project data into n+1 equal blocks; S103. Traverse through each element in the set K to find the key split point by getting the difference between the current distribution get the current distribution difference ; S104. Based on the maximum entropy theory, return the maximum distribution difference in step S103 , record the number of corresponding split points as , record the split points corresponding to , wherein indicates split points; S105. Backtrack the n value in step S102 to obtain the maximum distribution difference The corresponding n value is denoted as n represents the number of candidate split points. S106. According to the processing result of step S105 to step S106, a plurality of sub-domains with the most distribution diversity are acquired : ； S2. extracting hidden knowledge of each sub-domain by using a hybrid deep neural network model to realize extraction of knowledge to be transferred from the source project data; The step S2 comprises the following steps: S201. For the first subfield, knowledge containing spatial representation information is extracted using a convolutional neural network CNN, wherein ; S202. integrating knowledge containing spatial representation information by using a long short-term memory network LSTM to obtain hidden time-series information; S203. processing a degradation problem of the hybrid deep neural network model by using a skip connection of a residual network; S204. fusing the obtained hidden time-series information by using a fully connected layer, learning a mapping relationship between feature representation and true value, completing extraction of hidden knowledge of each sub-domain, and realizing extraction of knowledge to be transferred from the source project data; S3. fusing the extracted hidden knowledge of each sub-domain by using a multi-knowledge feature fusion method to obtain target project data and complete cross-project transfer of engineering equipment knowledge.

2. The engineered equipment knowledge cross-project transfer method of claim 1, wherein, The step S3 comprises the following steps: S301. Based on the mixed deep neural network model, by acquiring a plurality of sub-domain knowledge extraction models, the regression loss of the first sub-domain knowledge extraction model is obtained. S302. Freeze the feature extractor parameters of the subfield knowledge extraction model, and migrate them to the target domain, wherein the distribution difference knowledge of the high-level feature representation of each subfield and the target domain is obtained by extracting the first group of deep feature representations of the target domain based on the feature extractor of the first subfield knowledge extraction model. S302. Freeze the feature extractor parameters of the subfield knowledge extraction model, and migrate them to the target domain, wherein the distribution difference knowledge of the high-level feature representation of each subfield and the target domain is obtained by extracting the first group of deep feature representations of the target domain based on the feature extractor of the first subfield knowledge extraction model.​ S303. obtaining a multi-knowledge fusion feature extractor according to regression loss and distribution difference knowledge, inputting training data of a target domain, and obtaining high-level feature representation after multi-knowledge feature fusion; S304. splicing a fully connected layer after the multi-knowledge fusion feature extractor as a regressor of a target project prediction task, and constructing a prediction model of the target project based on the multi-knowledge fusion feature extractor and the regressor; S305. fine-tuning the prediction model of the target project according to the high-level feature representation after multi-knowledge feature fusion, and obtaining target project data by using the fine-tuned target project prediction model to complete cross-project transfer of engineering equipment knowledge.

3. The engineered equipment knowledge cross-project transfer method of claim 2, wherein, The first The expression of the regression loss of the first sub-domain knowledge extraction model is as follows: wherein, represents a regression loss of the th subfield knowledge extraction model, represents an output value of the regressor of the th subfield knowledge extraction model, represents a true value of the e th subfield, represents a regression layer of the th subfield knowledge extraction model, represents a deep feature representation of the th subfield extracted by the feature extractor of the th subfield knowledge extraction model, represents a training parameter of the regression layer of the th subfield knowledge extraction model, represents a feature extractor of the subfield knowledge extraction model, represents an input value of the e th subfield of the e th subfield knowledge extraction model, represents a training parameter of the feature extractor of the th subfield knowledge extraction model.

4. The engineered equipment knowledge cross-project transfer method of claim 3, wherein, An expression of the distribution difference knowledge of the high-level feature representation of each sub-domain and the target domain is as follows: wherein, denotes the distribution difference knowledge of the high-level feature representation of the th sub-domain and the target domain extracted by the feature extractor of the th sub-domain knowledge extraction model, denotes the MMD distance measure, denotes the feature representation of the e th sub-domain extracted by the e th sub-domain knowledge extraction model, denotes the feature representation of the target domain extracted by the e th sub-domain knowledge extraction model, denotes the input value of the target domain of the e th sub-domain knowledge extraction model, denotes the training parameter of the feature extractor of the e th sub-domain knowledge extraction model.

5. The engineered equipment knowledge cross-project transfer method of claim 4, wherein, An expression of the high-level feature representation after multi-knowledge feature fusion is as follows: wherein, represents the high-level feature representation after multi-knowledge feature fusion, represents an Add neural network layer in deep learning, having the effect of vector addition, represents and the output result after multiply layer processing, and respectively represent the training parameters of the layer and the training parameters of the layer, represents a Multiply neural network layer in deep learning, having the effect of matrix multiplication, represents the fusion weight of the feature extractor of the th sub-domain knowledge extraction model, represents the feature representation of the target domain extracted by the e th sub-domain knowledge extraction model, represents the normalization operation on the selected content.

6. The engineered equipment knowledge cross-project transfer method of claim 5, wherein, The step S305 is specifically: Fine-tuning the prediction model of the target project according to the high-level feature representation after multi-knowledge feature fusion by using the following formula, and obtaining target project data by using the fine-tuned target project prediction model to complete cross-project transfer of engineering equipment knowledge: wherein, represents an output value of the target item prediction model, represents a regressor as a target item prediction task, represents a training parameter of a regression layer of the multi-knowledge fusion knowledge extraction model.

7. An engineering equipment knowledge cross-project transfer system that performs the engineering equipment knowledge cross-project transfer method according to any one of claims 1 to 6, characterized by, It comprises: a source project time-series blocking module, configured to divide source project data into a plurality of sub-domains with the most distribution diversity by using an improved time-series blocking algorithm; a source project knowledge extraction module, configured to extract hidden knowledge of each sub-domain by using a hybrid deep neural network model to realize extraction of knowledge to be transferred from the source project data; a multi-knowledge-based feature fusion module, configured to fuse the extracted hidden knowledge of each sub-domain by using a multi-knowledge feature fusion method to obtain target project data and complete cross-project transfer of engineering equipment knowledge.

Images