Fusion method and system based on construction site multi-source heterogeneous data

Abstract

The invention discloses a fusion method and system based on construction site multi-source heterogeneous data, and belongs to the technical field of image processing. According to the method provided by the invention, after CNN neural network and Transform neural network feature extraction is carried out on multi-source heterogeneous data of a construction site, feature vectors of the heterogeneous source data are obtained, features extracted from the heterogeneous data are respectively input into respective MLP and Softmax layers to carry out construction specification model regression training, respective prediction results are obtained to generate local decisions, and the local decisions are subjected to construction specification model regression training. And using as input of decision fusion. And performing linear weighted summation on the prediction result to obtain a fusion result, and passing the fusion result through a Softmax layer to obtain a multi-source heterogeneous data fusion result. The fusion method is simple and has better expandability for various modes, and the most suitable analysis method can be adopted for different modes.

Description

Technical Field

[0001] The present invention belongs to the technical field of data processing and relates to a fusion method and system based on multi-source heterogeneous data of a construction site. Background Art

[0002] Multi-source heterogeneous data fusion is an information processing process that uses mathematical methods and technical tools to integrate data from multiple data sources such as sensors, systems or networks in a time series under certain criteria and automatically analyze, optimize and synthesize them to complete the required decision-making and estimation tasks. The perception sensors, audio and video equipment and text input tools at the construction site are the basis of data fusion, the perceived and acquired information is the object of data fusion processing, and the acquisition of heterogeneous data features and the training and learning of standardized construction models are the core of fusion.

[0003] Although construction sites currently have inspection and control for detection, identification, safety tools and machinery, and instruments, independent inspection and control easily form information islands, making it impossible to grasp accurate multi-dimensional information on the site as a whole. Summary of the invention

[0004] The purpose of the present invention is to solve the problem that independent detection and control in the prior art easily form information islands and cannot grasp the accurate multi-dimensional information of the site as a whole, and to provide a fusion method and system based on multi-source heterogeneous data of the construction site.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] The present invention proposes a fusion method based on multi-source heterogeneous data of a construction site, comprising:

[0007] Acquire multi-source heterogeneous construction site data;

[0008] Input the multi-source heterogeneous construction site data into the trained multi-source data fusion model to obtain the results of the construction in accordance with the current construction standards;

[0009] The trained multi-source data fusion model specifically includes: obtaining pre-processed multi-source heterogeneous data, building a multi-source data fusion model, sending the pre-processed multi-source heterogeneous data to the multi-source data fusion model training, and obtaining a trained multi-source data fusion model.

[0010] Preferably, the multi-source data fusion model includes a feature extraction module, a construction specification recognition module and a decision fusion module;

[0011] The feature extraction module extracts features from multi-source heterogeneous data, inputs the feature extraction results into the construction specification recognition module for feature recognition, and inputs the feature recognition results into the decision fusion module for feature prediction to obtain the construction site prediction results.

[0012] Preferably, the feature extraction module uses CNN and Transformer to extract features, and the specific implementation steps are as follows:

[0013] Shallow features are extracted through CNN. After extracting the shallow features, the Transformer model uses the shallow features output by CNN as input, and obtains the feature extraction results after multiple layers of convolution, pooling, self-attention calculation and feature splicing.

[0014] Preferably, the construction specification recognition module inputs the feature data into MLP for nonlinear operation to obtain hidden features, and obtains local decision results by Softmax for hidden features. The specific implementation steps are as follows:

[0015] The feature data X from the feature extraction module is input into the MLP. The MLP consists of several layers of fully connected networks. Each layer is processed by a nonlinear activation function to extract deep patterns in the feature data. For the input feature X, the output feature H(l) of the lth layer is calculated as follows:

[0016] H(l)=σ(W(l)H(l-1)+b(l))

[0017] Among them, H(0)=X is the input feature; W(l) and b(l) are the weight matrix and bias vector of the lth layer respectively; σ is the nonlinear activation function Leaky ReLU;

[0018] Through multi-layer nonlinear operations, MLP extracts the last layer of hidden features H(L) in the input feature data, and then uses the Softmax function to calculate the local decision result. The calculation formula is as follows:

[0019]

[0020] Among them, P i represents the predicted probability of the i-th category, which is the probability distribution of the current construction data in each construction specification category; is the i-th output value in the last layer of hidden features; C is the total number of all possible construction specification categories.

[0021] Preferably, the decision fusion module performs linear weighted sum processing on local decisions, and then inputs the processing results into Softmax to obtain the standardized construction prediction results. The specific implementation steps are as follows:

[0022] After being processed by the construction specification recognition module, N local decision results are obtained. Each decision result represents the probability distribution under different construction specification categories. The local decision is represented by a vector P (1) , P (2) ,…,P(N) , where each P (k) Contains the probability of C categories, in the form of:

[0023]

[0024] Set the weight of each local decision to α k , where k = 1, 2, ..., N, and satisfies the following constraints:

[0025]

[0026] Perform a linear weighted summation on all local decision results to generate the fused feature vector S, which is expressed as follows:

[0027]

[0028] After obtaining the fused feature vector S, the Softmax function is used to process it and the standardized construction prediction result Q = [Q1, Q2, ..., Q C ] is calculated as follows:

[0029]

[0030] Among them, Q represents the probability of the i-th category in the final prediction, which is the final construction specification prediction result; S i is the weighted sum score of the i-th category, each Q i The value represents the probability that the construction site data meets the i-th type of construction specifications. The specification category of the construction site is determined according to the maximum probability value in Q.

[0031] The present invention proposes a fusion system based on multi-source heterogeneous data of a construction site, comprising:

[0032] A data acquisition module, wherein the data acquisition module is used to acquire multi-source heterogeneous construction site data;

[0033] A data processing module, which is used to input multi-source heterogeneous construction site data into a trained multi-source data fusion model to obtain the results of the construction site construction in accordance with the current construction standards;

[0034] The trained multi-source data fusion model specifically includes: obtaining pre-processed multi-source heterogeneous data, building a multi-source data fusion model, sending the pre-processed multi-source heterogeneous data to the multi-source data fusion model training, and obtaining a trained multi-source data fusion model.

[0035] Preferably, the construction specification recognition module inputs the feature data into MLP for nonlinear operation to obtain hidden features, and obtains local decision results by Softmax for hidden features. The specific implementation steps are as follows:

[0036] The feature data X from the feature extraction module is input into the MLP. The MLP consists of several layers of fully connected networks. Each layer is processed by a nonlinear activation function to extract deep patterns in the feature data. For the input feature X, the output feature H(l) of the lth layer is calculated as follows:

[0037] H(l)=σ(W(l)H(l-1)+b(l))

[0038] Among them, H(0)=X is the input feature; W(l) and b(l) are the weight matrix and bias vector of the lth layer respectively; σ is the nonlinear activation function Leaky ReLU;

[0039] Through multi-layer nonlinear operations, MLP extracts the last layer of hidden features H(L) in the input feature data, and then uses the Softmax function to calculate the local decision result. The calculation formula is as follows:

[0040]

[0041] Among them, P i represents the predicted probability of the i-th category, which is the probability distribution of the current construction data in each construction specification category; is the i-th output value in the last layer of hidden features; C is the total number of all possible construction specification categories.

[0042] Preferably, the decision fusion module performs linear weighted sum processing on local decisions, and then inputs the processing results into Softmax to obtain the standardized construction prediction results. The specific implementation steps are as follows:

[0043] After being processed by the construction specification recognition module, N local decision results are obtained. Each decision result represents the probability distribution under different construction specification categories. The local decision is represented by a vector P (1) , P (2) ,…,P (N) , where each P (k) Contains the probability of C categories, in the form of:

[0044]

[0045] Set the weight of each local decision to α k , where k = 1, 2, ..., N, and satisfies the following constraints:

[0046]

[0047] Perform a linear weighted summation on all local decision results to generate the fused feature vector S, which is expressed as follows:

[0048]

[0049] After obtaining the fused feature vector S, the Softmax function is used to process it and the standardized construction prediction result Q = [Q1, Q2, ..., Q C ] is calculated as follows:

[0050]

[0051] Among them, Q represents the probability of the i-th category in the final prediction, which is the final construction specification prediction result; S i is the weighted sum score of the i-th category, each Q i The value represents the probability that the construction site data meets the i-th type of construction specifications. The specification category of the construction site is determined according to the maximum probability value in Q.

[0052] A terminal device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of a fusion method based on multi-source heterogeneous data of a construction site are implemented.

[0053] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of a fusion method based on multi-source heterogeneous data of a construction site.

[0054] Compared with the prior art, the present invention has the following beneficial effects:

[0055] The present invention proposes a fusion method based on multi-source heterogeneous data of a construction site. Since the construction site has monitoring requirements for working environment, identification, safety tools and machinery, and instruments, the present invention adopts different neural network models for different modalities to analyze different heterogeneous source data, uses the trained network model to extract features from the input source data, and then sends the extracted results to the respective models for fusion to form a local decision result, and finally uses the trained construction specification model to fuse the decision results of each heterogeneous source data to form the final decision result. The fusion method is simple and has better scalability for various modalities, and the most appropriate analysis method can be used for different modalities.

[0056] The present invention proposes a fusion system based on multi-source heterogeneous data of the construction site. By dividing the system into a data acquisition module and a data processing module, the result of the construction is obtained in accordance with the current construction standards, and the fusion of multi-source heterogeneous data of the construction site is realized. The modularization concept is adopted to make each module independent of each other, which is convenient for unified management of each module. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0058] Figure 1 This is a flow chart of the fusion method based on multi-source heterogeneous data at the construction site of the present invention.

[0059] Figure 2 This is a structural diagram of the multi-source data fusion model of the present invention.

[0060] Figure 3 This is a fusion system diagram based on multi-source heterogeneous data at the construction site of the present invention.

[0061] Figure 4 The figure is a schematic structural diagram of an electronic device of the present invention. DETAILED DESCRIPTION

[0062] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0063] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0064] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0065] In the description of the embodiments of the present invention, it should be noted that if the terms "upper", "lower", "horizontal", "inner", etc. indicate an orientation or positional relationship based on the orientation or positional relationship shown in the drawings, or the orientation or positional relationship in which the product of the invention is usually placed when in use, it is only for the convenience of describing the present invention and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention. In addition, the terms "first", "second", etc. are only used to distinguish the description, and cannot be understood as indicating or implying relative importance.

[0066] In addition, if the term "horizontal" appears, it does not mean that the component must be absolutely horizontal, but can be slightly tilted. For example, "horizontal" only means that its direction is more horizontal than "vertical", which does not mean that the structure must be completely horizontal, but can be slightly tilted.

[0067] In the description of the embodiments of the present invention, it is also necessary to explain that, unless otherwise clearly specified and limited, the terms "set", "install", "connect", and "connect" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal connection of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0068] The present invention is further described in detail below in conjunction with the accompanying drawings:

[0069] Example 1

[0070] Various sensor devices at the construction site are independent of each other and have different output data types. Therefore, the present invention proposes a fusion method based on multi-source heterogeneous data at the construction site, such as Figure 1 As shown, including:

[0071] S1. Acquire multi-source heterogeneous construction site data;

[0072] S2, input the multi-source heterogeneous construction site data into the trained multi-source data fusion model to obtain the results of the construction in accordance with the current construction standards;

[0073] The trained multi-source data fusion model specifically includes: obtaining pre-processed multi-source heterogeneous data, building a multi-source data fusion model, sending the pre-processed multi-source heterogeneous data to the multi-source data fusion model training, and obtaining a trained multi-source data fusion model.

[0074] The multi-source data fusion model includes a feature extraction module, a construction specification recognition module and a decision fusion module;

[0075] The feature extraction module extracts features from multi-source heterogeneous data, inputs the feature extraction results into the construction specification recognition module for feature recognition, and inputs the feature recognition results into the decision fusion module for feature prediction to obtain the construction site prediction results.

[0076] The feature extraction module uses CNN and Transformer to extract features. The specific implementation steps are as follows:

[0077] Shallow features are extracted through CNN. After extracting the shallow features, the Transformer model uses the shallow features output by CNN as input, and obtains the feature extraction results after multiple layers of convolution, pooling, self-attention calculation and feature splicing.

[0078] The construction specification recognition module inputs the feature data into MLP for nonlinear operation to obtain hidden features, and obtains local decision results by Softmax for hidden features. The specific implementation steps are as follows:

[0079] The feature data X from the feature extraction module is input into the MLP. The MLP consists of several layers of fully connected networks. Each layer is processed by a nonlinear activation function to extract deep patterns in the feature data. For the input feature X, the output feature H(l) of the lth layer is calculated as follows:

[0080] H(l)=σ(W(l)H(l-1)+b(l))

[0081] Where H(0)=X is the input feature; W(l) and b(l) are the weight matrix and bias vector of the lth layer respectively; σ is the nonlinear activation function Leaky ReLU;

[0082] Through multi-layer nonlinear operations, MLP extracts the last layer of hidden features H(L) in the input feature data, and then uses the Softmax function to calculate the local decision result. The calculation formula is as follows:

[0083]

[0084] Among them, P i It represents the predicted probability of the i-th category and is the probability distribution of the current construction data in each construction specification category; is the i-th output value in the last layer of hidden features; C is the total number of all possible construction specification categories.

[0085] The decision fusion module performs linear weighted sum processing on local decisions, and then inputs the processing results into Softmax to obtain the standardized construction prediction results. The specific implementation steps are as follows:

[0086] After being processed by the construction specification recognition module, N local decision results are obtained. Each decision result represents the probability distribution under different construction specification categories. The local decision is represented by a vector P (1) , P (2) ,…,P (N) , where each P (k) Contains the probability of C categories, in the form of:

[0087]

[0088] Set the weight of each local decision to α k , where k = 1, 2, ..., N, and satisfies the following constraints:

[0089]

[0090] Perform a linear weighted summation on all local decision results to generate the fused feature vector S, which is expressed as follows:

[0091]

[0092] After obtaining the fused feature vector S, the Softmax function is used to process it and the standardized construction prediction result Q = [Q1, Q2, ..., Q C ] is calculated as follows:

[0093]

[0094] Among them, Q represents the probability of the i-th category in the final prediction, which is the final construction specification prediction result; S i is the weighted sum score of the i-th category, each Q i The value represents the probability that the construction site data meets the i-th type of construction specifications. The specification category of the construction site is determined according to the maximum probability value in Q.

[0095] The present invention proposes a fusion method based on multi-source heterogeneous data of construction sites, such as Figure 2 As shown in the figure, after extracting features from the multi-source heterogeneous data of the construction site using CNN neural network and Transformer neural network, the feature vectors of each heterogeneous source data are obtained, and the features extracted from the heterogeneous data are input into the respective MLP and Softmax layers for construction specification model regression training, and the respective prediction results are obtained to generate local decisions, which are used as input for decision fusion. After that, the prediction results are linearly weighted summed to obtain the fusion result, and then the fusion result is passed through the Softmax layer to obtain the result of multi-source heterogeneous data fusion.

[0096] The following describes the multi-source data fusion model in detail based on the results:

[0097] The multi-source data fusion model includes feature extraction module, construction specification recognition module and decision fusion module;

[0098] The feature extraction module uses CNN and Transformer for feature extraction. First, shallow features are extracted through convolutional neural network CNN. The main advantage of CNN is its ability to understand spatial structure, and it is particularly suitable for processing images, videos and other data information with spatial relationships. In this method, CNN is used to process video data and high-frequency sensor data from the construction site, such as time series data of vibration and temperature sensors. Through multi-layer convolution operations, CNN can identify feature patterns in image or signal data and extract detailed information such as object contours, edges, textures, etc. The feature extraction process is completed by a combination of multi-layer convolution and pooling operations, and features at different levels are extracted layer by layer, so that high-level features can effectively represent complex patterns in the data, laying the foundation for subsequent feature fusion. After extracting shallow features, the present invention uses the Transformer module to capture long-distance dependencies and global contextual relationships. The multi-head self-attention mechanism of Transformer can effectively process complex spatiotemporal relationships from different data sources. In the present invention, Transformer processes time series data and structured information from multiple sensors in parallel to identify potential relationships between data. The Transformer model takes the local features output by CNN as input, and assigns weights to different features through a multi-head attention layer to obtain an integrated expression of global information. In order to further optimize the effect of feature extraction, the present invention adopts a hierarchical feature fusion method to integrate the features extracted by CNN and Transformer layer by layer. During the integration process, the local features extracted by CNN are embedded into the input sequence of Transformer, thereby taking into account the interactive relationship between local and global features. After multiple layers of convolution, pooling, self-attention calculation and feature splicing, a comprehensive feature representation is finally formed. This representation not only includes the local detail features of each sensor and image data at the construction site, but also includes the global spatiotemporal dependency, providing reliable data support for construction specification recognition and subsequent decision fusion.

[0099] The specific implementation steps of the construction specification recognition module are as follows: First, the feature data X from the feature extraction module is input into the MLP. The MLP consists of several layers of fully connected networks, each of which is processed by a nonlinear activation function to extract deep patterns in the feature data. For the input feature X, the calculation process of the lth layer is as follows:

[0100] H(l)=σ(W(l)H(l-1)+b(l))

[0101] Among them, H(0)=X is the input feature; W(l) and b(l) are the weight matrix and bias vector of the lth layer respectively; σ is the nonlinear activation function Leaky ReLU. Through multiple layers of nonlinear operations, MLP can extract hidden features in the input feature data. These hidden features represent the deep feature relationships in the construction data, thereby more accurately characterizing the construction status on site. After obtaining the last layer of hidden features H(L), the Softmax function is used to calculate the local decision results. The Softmax function is used to convert hidden features into probability distribution results that meet construction specifications. The calculation formula is as follows:

[0102]

[0103] Among them, P i represents the predicted probability of the i-th category; is the i-th output value in the last layer of hidden features; C is the total number of all possible construction specification categories. Softmax operation result P i This is the local decision result, which indicates the probability distribution of the current construction data in each construction specification category. The construction specification identification module determines the data as the most suitable specification category based on the probability distribution. This step can effectively identify the specification status in complex construction sites through the combination of MLP and Softmax, providing a reliable basis for subsequent decision fusion.

[0104] The processing of the decision fusion module includes the following steps: first, linear weighted summation of local decisions is performed, and then the Softmax function is used to calculate the final probability distribution of standard construction.

[0105] Assume that after processing by the construction specification recognition module, N local decision results are obtained, and each decision result represents the probability distribution under different construction specification categories. These local decisions can be represented as vector P (1) , P (2) ,…,P (N) , where each P (k) Contains the probability of C categories, in the form of:

[0106]

[0107] The decision fusion module assigns different weights to each local decision to reflect the importance of different data sources in the decision. The weight of each local decision is set to α k , where k = 1, 2, ..., N, and satisfies the following constraints:

[0108]

[0109] Perform a linear weighted summation on all local decision results to generate the fused feature vector S, which is expressed as follows:

[0110]

[0111] Here, S=[S1,S2,…,S C ] represents the weighted score of each construction specification category. After obtaining the weighted summation result S, in order to convert it into the probability distribution of the construction specification, the Softmax function is used for processing. The final specification construction prediction result Q = [Q1, Q2, ..., Q C ] is calculated as follows:

[0112]

[0113] Among them: Q represents the probability of the i-th category in the final prediction; S i is the weighted sum score of the i-th category. The vector Q output by Softmax represents the final construction specification prediction result. Each Q i The value represents the probability that the construction site data meets the i-th type of construction specification. According to the maximum probability value in Q, the specification category of the construction site can be determined to ensure that the output results are most in line with the actual construction requirements. Through the linear weighted summation and Softmax calculation of this decision fusion module, the system can integrate the local decisions of multi-source data and generate accurate construction specification prediction results, which helps to standardize construction site management and improve the safety and efficiency of construction.

[0114] The decision-level feature fusion method of multi-source heterogeneous data at the construction site of the present invention has good anti-interference ability, real-time and fault tolerance, and is particularly suitable for the prediction and judgment of standardized construction at the construction site. The construction site has monitoring requirements for the working environment, identification, safety tools and machinery, and instruments. The present invention adopts different neural network models for analysis for different modes, adopts Transformer neural network to extract features for this type of data decision analysis for the environment, adopts CNN neural network to extract features and this type of data decision analysis for audio, video and text, and then adopts the construction specification decision model to fuse the results of various data decision analysis to form a global decision. The core of the present invention is to use different neural network models to analyze different heterogeneous source data, use the trained network model to input source data for feature extraction, and then send the extraction results to their respective models for fusion to form a local decision result, and finally use the trained construction specification model to fuse the decision results of each heterogeneous source data to form a final decision result. The fusion method is simple and has better scalability for various modes, and the most suitable analysis method can be used for different modes.

[0115] The method proposed by the present invention has the following advantages:

[0116] 1) Provide more comprehensive and accurate information: Multi-source heterogeneous data can make up for the shortcomings of a single data source. The data obtained through data fusion has more dimensions and more complete information, which can provide more comprehensive and accurate data support and help improve the accuracy of decision-making.

[0117] 2) Reduce data processing costs: The use of multi-source heterogeneous data fusion can avoid information islands. By combining different data sources, more effective and high-quality data can be obtained, which can reduce the cost and difficulty of data analysis.

[0118] 3) Improve data processing efficiency: In the process of data fusion, use a variety of parallel computing technologies to simplify data processing complexity and effectively improve processing speed and efficiency.

[0119] 4) Support multi-dimensional application requirements: Multi-source heterogeneous data fusion provides better support for multiple and multi-dimensional application requirements of data. Data fusion technology can support complex application requirements.

[0120] Example 2

[0121] The present invention proposes a fusion system based on multi-source heterogeneous data of construction sites, such as Figure 3 As shown, including:

[0122] A data acquisition module, wherein the data acquisition module is used to acquire multi-source heterogeneous construction site data;

[0123] A data processing module, which is used to input multi-source heterogeneous construction site data into a trained multi-source data fusion model to obtain the results of the construction site construction in accordance with the current standards;

[0124] The trained multi-source data fusion model specifically includes: obtaining pre-processed multi-source heterogeneous data, building a multi-source data fusion model, sending the pre-processed multi-source heterogeneous data to the multi-source data fusion model training, and obtaining a trained multi-source data fusion model.

[0125] The multi-source data fusion model includes a feature extraction module, a construction specification recognition module and a decision fusion module;

[0126] The feature extraction module extracts features from multi-source heterogeneous data, inputs the feature extraction results into the construction specification recognition module for feature recognition, and inputs the feature recognition results into the decision fusion module for feature prediction to obtain the construction site prediction results.

[0127] The feature extraction module uses CNN and Transformer to extract features. The specific implementation steps are as follows:

[0128] Shallow features are extracted through CNN. After extracting the shallow features, the Transformer model uses the shallow features output by CNN as input, and obtains the feature extraction results after multiple layers of convolution, pooling, self-attention calculation and feature splicing.

[0129] The construction specification recognition module inputs the feature data into MLP for nonlinear operation to obtain hidden features, and obtains local decision results by Softmax for hidden features. The specific implementation steps are as follows:

[0130] The feature data X from the feature extraction module is input into the MLP. The MLP consists of several layers of fully connected networks. Each layer is processed by a nonlinear activation function to extract deep patterns in the feature data. For the input feature X, the output feature H(l) of the lth layer is calculated as follows:

[0131] H(l)=σ(W(l)H(l-1)+b(l))

[0132] Where H(0)=X is the input feature; W(l) and b(l) are the weight matrix and bias vector of the lth layer respectively; σ is the nonlinear activation function Leaky ReLU;

[0133] Through multi-layer nonlinear operations, MLP extracts the last layer of hidden features H(L) in the input feature data, and then uses the Softmax function to calculate the local decision result. The calculation formula is as follows:

[0134]

[0135] Among them, P i It represents the predicted probability of the i-th category and is the probability distribution of the current construction data in each construction specification category; is the i-th output value in the last layer of hidden features; C is the total number of all possible construction specification categories.

[0136] The decision fusion module performs linear weighted sum processing on local decisions, and then inputs the processing results into Softmax to obtain the standardized construction prediction results. The specific implementation steps are as follows:

[0137] After being processed by the construction specification recognition module, N local decision results are obtained. Each decision result represents the probability distribution under different construction specification categories. The local decision is represented by a vector P (1) , P (2) ,…,P (N) , where each P (k) Contains the probability of C categories, in the form of:

[0138]

[0139] Set the weight of each local decision to α k , where k = 1, 2, ..., N, and satisfies the following constraints:

[0140]

[0141] Perform a linear weighted summation on all local decision results to generate the fused feature vector S, which is expressed as follows:

[0142]

[0143] After obtaining the fused feature vector S, the Softmax function is used to process it and the standardized construction prediction result Q = [Q1, Q2, ..., Q C ] is calculated as follows:

[0144]

[0145] Among them, Q represents the probability of the i-th category in the final prediction, which is the final construction specification prediction result; S i is the weighted sum score of the i-th category, each Q i The value represents the probability that the construction site data meets the i-th type of construction specifications. The specification category of the construction site is determined according to the maximum probability value in Q.

[0146] Example 3

[0147] See also Figure 4 As shown, the present invention also provides an electronic device 100 based on a fusion method of multi-source heterogeneous data at a construction site; the electronic device 100 includes a memory 101, at least one processor 102, a computer program 103 stored in the memory 101 and executable on the at least one processor 102, and at least one communication bus 104.

[0148] The memory 101 can be used to store the computer program 103. The processor 102 implements the steps of the fusion method based on multi-source heterogeneous data of the construction site described in Example 1 by running or executing the computer program stored in the memory 101 and calling the data stored in the memory 101. The memory 101 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data (such as audio data) created according to the use of the electronic device 100, etc. In addition, the memory 101 can include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other non-volatile solid-state storage devices.

[0149] The at least one processor 102 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The processor 102 may be a microprocessor or any conventional processor, etc. The processor 102 is the control center of the electronic device 100, and uses various interfaces and lines to connect various parts of the entire electronic device 100.

[0150] The memory 101 in the electronic device 100 stores a plurality of instructions to implement a method for integrating multi-source heterogeneous data based on a construction site, and the processor 102 can execute the plurality of instructions to implement:

[0151] Acquire multi-source heterogeneous construction site data;

[0152] Input the multi-source heterogeneous construction site data into the trained multi-source data fusion model to obtain the results of the construction in accordance with the current construction standards;

[0153] The trained multi-source data fusion model specifically includes: obtaining pre-processed multi-source heterogeneous data, building a multi-source data fusion model, sending the pre-processed multi-source heterogeneous data to the multi-source data fusion model training, and obtaining a trained multi-source data fusion model.

[0154] Example 4

[0155] If the module / unit integrated in the electronic device 100 is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the process in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device that can carry the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory and read-only memory (ROM, Read-Only Memory).

[0156] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0157] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0158] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1A function specified in one or more boxes.

[0159] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0160] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A fusion method based on multi-source heterogeneous data of construction sites, characterized in that: include: Acquire multi-source heterogeneous construction site data; Input the multi-source heterogeneous construction site data into the trained multi-source data fusion model to obtain the results of the construction in accordance with the current construction standards; The trained multi-source data fusion model specifically includes: obtaining pre-processed multi-source heterogeneous data, building a multi-source data fusion model, sending the pre-processed multi-source heterogeneous data to the multi-source data fusion model training, and obtaining a trained multi-source data fusion model.

2. The fusion method based on multi-source heterogeneous data of the construction site according to claim 1 is characterized in that: The multi-source data fusion model includes a feature extraction module, a construction specification recognition module and a decision fusion module; The feature extraction module extracts features from multi-source heterogeneous data, inputs the feature extraction results into the construction specification recognition module for feature recognition, and inputs the feature recognition results into the decision fusion module for feature prediction to obtain the construction site prediction results.

3. The fusion method based on multi-source heterogeneous data of the construction site according to claim 2 is characterized in that: The feature extraction module uses CNN and Transformer to extract features. The specific implementation steps are as follows: Shallow features are extracted through CNN. After extracting the shallow features, the Transformer model uses the shallow features output by CNN as input, and obtains the feature extraction results after multiple layers of convolution, pooling, self-attention calculation and feature splicing.

4. The fusion method based on multi-source heterogeneous data of the construction site according to claim 2 is characterized in that: The construction specification recognition module inputs the feature data into MLP for nonlinear operation to obtain hidden features, and obtains local decision results by Softmax for hidden features. The specific implementation steps are as follows: The feature data X from the feature extraction module is input into the MLP. The MLP consists of several layers of fully connected networks. Each layer is processed by a nonlinear activation function to extract deep patterns in the feature data. For the input feature X, the output feature H(l) of the lth layer is calculated as follows: H(l)=σ(W(l)H(l-1)+b(l)) Where H(0)=X is the input feature; W(l) and b(l) are the weight matrix and bias vector of the lth layer respectively; σ is the nonlinear activation function Leaky ReLU; Through multi-layer nonlinear operations, MLP extracts the last layer of hidden features H(L) in the input feature data, and then uses the Softmax function to calculate the local decision result. The calculation formula is as follows: Among them, P i It represents the predicted probability of the i-th category and is the probability distribution of the current construction data in each construction specification category; is the i-th output value in the last layer of hidden features; C is the total number of all possible construction specification categories.

5. The fusion method based on multi-source heterogeneous data of the construction site according to claim 2 is characterized in that: The decision fusion module performs linear weighted sum processing on local decisions, and then inputs the processing results into Softmax to obtain the standardized construction prediction results. The specific implementation steps are as follows: After being processed by the construction specification recognition module, N local decision results are obtained. Each decision result represents the probability distribution under different construction specification categories. The local decision is represented by a vector P (1) , P (2) ,…,P (N) , where each P (k) Contains the probability of C categories, in the form of: Set the weight of each local decision to α k , where k = 1, 2, ..., N, and satisfies the following constraints: Perform a linear weighted summation on all local decision results to generate the fused feature vector S, which is expressed as follows: After obtaining the fused feature vector S, the Softmax function is used to process it and the standardized construction prediction result Q = [Q1, Q2, ..., Q C ] is calculated as follows: Among them, Q represents the probability of the i-th category in the final prediction, which is the final construction specification prediction result; S i is the weighted sum score of the i-th category, each Q i The value represents the probability that the construction site data meets the i-th type of construction specifications. The specification category of the construction site is determined according to the maximum probability value in Q.

6. A fusion system based on multi-source heterogeneous data at a construction site, characterized in that: include: A data acquisition module, wherein the data acquisition module is used to acquire multi-source heterogeneous construction site data; A data processing module, which is used to input multi-source heterogeneous construction site data into a trained multi-source data fusion model to obtain the results of the construction site construction in accordance with the current standards; The trained multi-source data fusion model specifically includes: obtaining pre-processed multi-source heterogeneous data, building a multi-source data fusion model, sending the pre-processed multi-source heterogeneous data to the multi-source data fusion model training, and obtaining a trained multi-source data fusion model.

7. The fusion system based on multi-source heterogeneous data of the construction site according to claim 6 is characterized in that: The construction specification recognition module inputs the feature data into MLP for nonlinear operation to obtain hidden features, and obtains local decision results by Softmax for hidden features. The specific implementation steps are as follows: The feature data X from the feature extraction module is input into the MLP. The MLP consists of several layers of fully connected networks. Each layer is processed by a nonlinear activation function to extract deep patterns in the feature data. For the input feature X, the output feature H(l) of the lth layer is calculated as follows: H(l)=σ(W(l)H(l-1)+b(l)) Where H(0)=X is the input feature; W(l) and b(l) are the weight matrix and bias vector of the lth layer respectively; σ is the nonlinear activation function Leaky ReLU; Through multi-layer nonlinear operations, MLP extracts the last layer of hidden features H(L) in the input feature data, and then uses the Softmax function to calculate the local decision result. The calculation formula is as follows: Among them, P i It represents the predicted probability of the i-th category and is the probability distribution of the current construction data in each construction specification category; is the i-th output value in the last layer of hidden features; C is the total number of all possible construction specification categories.

8. The fusion system based on multi-source heterogeneous data of the construction site according to claim 6 is characterized in that: The decision fusion module performs linear weighted sum processing on local decisions, and then inputs the processing results into Softmax to obtain the standardized construction prediction results. The specific implementation steps are as follows: After being processed by the construction specification recognition module, N local decision results are obtained. Each decision result represents the probability distribution under different construction specification categories. The local decision is represented by a vector P (1) , P (2) ,…,P (N) , where each P (k) Contains the probability of C categories, in the form of: Set the weight of each local decision to α k , where k = 1, 2, ..., N, and satisfies the following constraints: Perform a linear weighted summation on all local decision results to generate the fused feature vector S, which is expressed as follows: After obtaining the fused feature vector S, the Softmax function is used to process it and the standardized construction prediction result Q = [Q1, Q2, ..., Q C ] is calculated as follows: Among them, Q represents the probability of the i-th category in the final prediction, which is the final construction specification prediction result; S i is the weighted sum score of the i-th category, each Q i The value represents the probability that the construction site data meets the i-th type of construction specifications. The specification category of the construction site is determined according to the maximum probability value in Q.

9. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the fusion method based on multi-source heterogeneous data of the construction site as described in any one of claims 1 to 5 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the fusion method based on multi-source heterogeneous data of the construction site are implemented as described in any one of claims 1 to 5.

Images