Construction process quality monitoring method and system based on multi-modal data

Abstract

The invention relates to the technical field of construction quality monitoring, in particular to a construction process quality monitoring method and system based on multi-modal data, and the method comprises the steps: S1, carrying out the omnibearing sensing of a construction pavement region through a multi-modal sensing system, and obtaining real-time monitoring data; and S2, preprocessing the real-time monitoring data and finishing time alignment. When the method is used, through multi-source data acquisition, fusion processing and construction and application of a digital twin model, the state of data splitting in monitoring, analysis and control links in traditional construction monitoring is broken, and the construction efficiency is improved. Full-automatic response from data acquisition to process adjustment is realized, the manual intervention cost is reduced by 70%, the defect identification accuracy is improved to 95% through combination of cross validation of multi-modal data, visual inspection, equipment parameter monitoring and simulation prediction results, the control accuracy of the construction process is enhanced, the occurrence rate of quality problems is reduced by 60%, and the construction efficiency is improved. And the quality control precision is improved.

Description

Technical Field

[0001] This invention relates to the field of construction quality monitoring technology, specifically to a construction process quality monitoring method and system based on multimodal data. Background Technology

[0002] Traditional road construction management relies heavily on manual experience and on-site supervision, which suffers from numerous problems, including untimely and inaccurate information transmission, lack of data accumulation and analysis, difficulty in precise control of the construction process, and difficulty in tracing quality issues. To improve the quality, efficiency, and safety of road construction and achieve sustainable development, there is an urgent need to introduce intelligent management and control technologies. Domestic research on the application of drones in highway construction inspection is still in its early stages, mainly focusing on simple pavement defect detection and preliminary measurement of some parameters. Applications in road construction inspection are relatively limited, and a mature technical system and standards have not yet been formed.

[0003] Patent publication number CN118396478A describes in its specification that "This invention relates to the field of highway pavement quality testing technology, and particularly to a method for highway pavement quality testing for engineering supervision. The method includes the following steps: dynamically determining the monitoring area based on construction design drawings and engineering requirements to obtain monitoring area data; determining multi-dimensional checkpoints based on the monitoring area data to obtain monitoring area checkpoint data; measuring the density of the monitoring area checkpoint data using a kernel density meter to obtain checkpoint density data; and extracting density features from the checkpoint density data to obtain checkpoint density feature data, for use in highway pavement quality testing for engineering supervision." The invention achieves scientific, efficient, and accurate pavement quality testing, comprehensively ensuring highway construction quality, improving engineering supervision efficiency, and providing transparent and traceable quality management. While the aforementioned technology achieves scientific and efficient pavement quality testing through dynamically determining monitoring areas, multi-dimensionally arranging checkpoints, and using a nuclear density meter to measure density and extract features, thus improving engineering supervision efficiency and quality assurance capabilities, traditional methods in pavement construction management rely on manual experience and on-site supervision. This results in a disconnect between monitoring, analysis, and control, leading to delayed and inaccurate information transmission, a lack of data accumulation and in-depth analysis, and difficulty in precisely controlling the construction process.

[0004] In conclusion, developing a construction process quality monitoring method and system based on multimodal data remains a critical issue that urgently needs to be addressed in the field of construction quality monitoring technology. Summary of the Invention

[0005] The purpose of this invention is to address the shortcomings of existing technologies in road construction management, which rely on manual experience and on-site supervision, resulting in delayed and inaccurate information transmission, lack of data accumulation and in-depth analysis, and difficulty in accurately controlling the construction process. This invention provides a construction process quality monitoring method and system based on multimodal data.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] This invention provides a construction process quality monitoring method based on multimodal data, comprising:

[0008] S1. The construction road surface area is perceived in all directions through a multimodal sensing system to obtain real-time monitoring data;

[0009] S2. The real-time monitoring data is preprocessed and time-aligned. Simultaneously, spatiotemporal calibration technology and multimodal fusion model are used to generate data feature vectors.

[0010] S3. Construct a digital twin model of the construction process based on the data feature vectors, perform simulation and optimization evaluation, and output the simulation results;

[0011] S4. Based on the real-time monitoring data and simulation results, perform comparative analysis, dynamically monitor road surface quality indicators, output monitoring results, and set up an intelligent early warning mechanism.

[0012] Furthermore, in step S1, the method for obtaining real-time monitoring data by using a multimodal sensing system to perform omnidirectional sensing of the construction pavement area is as follows:

[0013] The process involves a multimodal perception system to comprehensively perceive the construction pavement area. UAVs equipped with optical cameras, infrared thermal imagers, and lidar are used to acquire images, temperature fields, and 3D terrain data. Simultaneously, IoT monitoring units are deployed on key equipment, including but not limited to mixing plants, pavers, rollers, and transport vehicles, to collect their operating parameters. Environmental sensors are also deployed at the construction site to obtain environmental data such as temperature, humidity, and wind speed. The sampling points of the 3D terrain data, key equipment operating parameters, and environmental data are then aligned using a Gaussian kernel weighting function and a BeiDou positioning system timestamp. The expression is:

[0014]

[0015] In the formula, κ ain (a,b) represents the alignment similarity function, where the subscript ain indicates that this is a kernel function used for alignment, outputting a similarity value between 0 and 1 for a and b, ‖ab‖. 2Let (a, b) represent the squared Euclidean distance between vectors a and b, and σ represent the standard deviation of the Gaussian kernel. 2 It is twice the square of the standard deviation, and exp(·) represents the exponent with the natural logarithm as the base, thus obtaining real-time monitoring data.

[0016] Further, in step S2, the real-time monitoring data is preprocessed and time-aligned. Simultaneously, the data feature vector is generated using spatiotemporal calibration technology and a multimodal fusion model.

[0017] The real-time monitoring data is preprocessed and time-aligned. A bilateral filter is introduced to denoise and enhance the 3D terrain data within the real-time monitoring data. CNN feature extraction is then used to extract road surface features such as cracks, potholes, and smoothness. The expression is:

[0018]

[0019] In the formula, z i This represents the value of the i-th point in the original data. It is the new value of the i-th point after bilateral filtering, z j p represents the original value of the j-th neighbor node. i p represents the spatial location of point i. j Represents the spatial position of point j, ||p i -p j || 2 Let z represent the square of the Euclidean distance between points i and j in space. i -z j ) 2 Let represent the square of the numerical difference between points i and j. The standard deviation of spatial distance The standard deviation represents the numerical difference, and exp(·) represents the exponential function. Let W represent the neighborhood set of point i. i The normalization factor represents the unified format and normalization of key equipment operating parameters and environmental data in the real-time monitoring data, and completes time alignment. The spatiotemporal calibration technology and multimodal fusion model are used to achieve effective correlation between point data and surface data through the spatiotemporal calibration technology. The multimodal fusion model is based on the Transformer architecture and integrates the multimodal information of the three-dimensional terrain data, key equipment operating parameters and environmental data to generate a unified data feature vector.

[0020] Furthermore, in step S3, the method for constructing a digital twin model of the construction process based on the data feature vectors, performing simulation and optimization evaluation, and outputting the simulation results is as follows:

[0021] The digital twin model utilizes a point cloud generated by LiDAR to construct a continuous terrain function through surface fitting, expressed as:

[0022]

[0023] In the formula, Indicates all fitting coefficients α kl Find the set of values ​​that minimizes the objective function. This indicates that the summation is performed on all N sampling points. Represents the corresponding point (x) i ,y i The fitted value of ) Let represent the magnitude of the error, λ be the regularization coefficient, and ∫…dxdy represent the integral over the entire two-dimensional space. Let x represent the second derivative of the surface. Let represent the second derivative of the surface y. z represents the sum and square of all curvatures. i This represents the value of the i-th point in the original data. This represents the result of fitting the function at position (x, y) on a two-dimensional plane. This indicates that the index k is incremented from 0 to K. This indicates that index l is incremented from 0 to L, α kl Represents the fitting coefficient. Represents the k-th element with respect to x. spline basis functions Represents the l-th term with respect to y. Spline basis functions use a high-precision 3D terrain model constructed by lidar as a spatial reference, and equipment operating parameters and environmental data as dynamic inputs. By embedding a physical modeling engine and multi-objective optimization algorithm, they dynamically drive the virtual construction process and map and predict the trend of construction quality changes in real time, including but not limited to the response relationship between paving speed, number of compaction passes and compaction degree.

[0024] Furthermore, in step S3, the method for constructing a digital twin model of the construction process based on the data feature vectors, performing simulation and optimization evaluation, and outputting the simulation results is as follows:

[0025] The digital twin model is used to conduct simulation and optimization evaluation of the combination of equipment operating parameters and environmental data, calculate the minimum paving time, the standard deviation of compaction degree, and the power consumption per unit area, and form a multi-objective optimization problem expression:

[0026]

[0027] In the formula, θ represents the set of parameters optimized in the digital twin model, and v pP(t) is the forward speed of the paver at time t, P(t) is the total power of all key construction equipment at time t, n(t) is the number of times the roller compacts the current area at time t, and Θ represents the feasible region of the parameters. The objective is to minimize these three objective functions. This indicates the penalty target for the speed of paving. Indicates the standard deviation of compaction. This represents the energy consumption cost per unit area. This indicates that the time integral covers the entire road construction period from time 0 to time T. Ω represents the reciprocal of the paver's forward velocity at time t, and Ω represents the two-dimensional spatial extent of the construction area. Ω …dxdy represents the area integral of the variable over the entire construction area Ω, ξ total (x,y,T) represents the compaction value measured at position x,y and time T. This indicates the average compaction degree of the area. This represents the square of the difference between the compaction degree at each point and the average value. This represents the summation of all M pieces of equipment at the road construction site, v k (t) represents the operating speed of the k-th device at time t, P k (t)·v k (t) is equivalent to the energy output per unit time multiplied by the processing rate, and the simulation result is output.

[0028] Furthermore, in step S4, the method for comparing and analyzing the real-time monitoring data with the simulation results, dynamically monitoring road surface quality indicators, outputting monitoring results, and setting up an intelligent early warning mechanism is as follows:

[0029] When the monitoring results indicate quality risks including, but not limited to, insufficient compaction and abnormal paving thickness, the set intelligent early warning mechanism is automatically triggered. A risk discrimination function is defined using discrimination logic, all risk items are combined into a risk vector, and a comprehensive early warning intensity score is defined, expressed as:

[0030]

[0031] In the formula, S aet (x,y,t) represents the warning score at coordinates x,y and time t. This represents a weighted summation of K risk factors, γ k R represents the weighting coefficient for the k-th risk. k (x,y,t) represents the risk score value of the k-th risk factor at coordinates x,y and time t. Based on the digital twin model, multiple adjustment schemes are simulated, and the optimal scheme is selected and fed back to the external device system through a closed loop control command.

[0032] Furthermore, in step S4, the method for comparing and analyzing the real-time monitoring data with the simulation results, dynamically monitoring road surface quality indicators, outputting monitoring results, and setting up an intelligent early warning mechanism is as follows:

[0033] After the adjustment plan is implemented, the drone will conduct a follow-up inspection of the corrected area to collect inspection data.

[0034]

[0035] In the formula, X cek (x,y) represents the vector of road surface quality parameters obtained at the coordinate point (x,y) after on-site inspection by a drone. This represents the compaction degree measured after a re-check at the coordinate point (x, y). This represents the paving thickness measured after verification at coordinate point (x, y).

[0036] (x,y) represents the flatness index measured after verification at coordinate point (x,y), and the corrected validity function is calculated. The expression is:

[0037] η(x,y)=exp(-‖X cek (x,y)-X tgt ‖2)

[0038] In the formula, η(x,y) represents the corrected validity score at the two-dimensional coordinate point (x,y), with a value range of (0,1], and is a natural exponential function. tgt The vector represents the ideal index value of the expected target, and ||·|2 represents the Euclidean norm. The state of the digital twin model is updated based on the review data, and all construction process data is uniformly stored in the external data platform and data lake system.

[0039] On the other hand, the present invention also provides a construction process quality monitoring system based on multimodal data, comprising:

[0040] The data acquisition module uses a multimodal sensing system to perceive the construction road surface area from all angles and obtain real-time monitoring data.

[0041] The data fusion module preprocesses the real-time monitoring data and performs time alignment, while using spatiotemporal calibration technology and a multimodal fusion model to generate data feature vectors.

[0042] The digital twin module constructs a digital twin model of the construction process based on the data feature vectors, performs simulation and optimization evaluation, and outputs simulation results;

[0043] The monitoring and early warning module compares and analyzes the real-time monitoring data with the simulation results, dynamically monitors road surface quality indicators, outputs monitoring results, and sets up an intelligent early warning mechanism.

[0044] Furthermore, the operation process of the data acquisition module includes:

[0045] The process involves a multimodal perception system to comprehensively perceive the construction pavement area. UAVs equipped with optical cameras, infrared thermal imagers, and lidar are used to acquire images, temperature fields, and 3D terrain data. Simultaneously, IoT monitoring units are deployed on key equipment, including but not limited to mixing plants, pavers, rollers, and transport vehicles, to collect their operating parameters. Environmental sensors are also deployed at the construction site to obtain environmental data such as temperature, humidity, and wind speed. The sampling points of the 3D terrain data, key equipment operating parameters, and environmental data are then aligned using a Gaussian kernel weighting function and a BeiDou positioning system timestamp. The expression is:

[0046]

[0047] In the formula, κ ain (a,b) represents the alignment similarity function, where the subscript ain indicates that this is a kernel function used for alignment, outputting a similarity value between 0 and 1 for a and b, ‖ab‖. 2 Let (a, b) represent the squared Euclidean distance between vectors a and b, and σ represent the standard deviation of the Gaussian kernel. 2 It is twice the square of the standard deviation, and exp(·) represents the exponent with the natural logarithm as the base, thus obtaining real-time monitoring data;

[0048] Furthermore, the operation process of the data fusion module includes:

[0049] The real-time monitoring data is preprocessed and time-aligned. A bilateral filter is introduced to denoise and enhance the 3D terrain data within the real-time monitoring data. CNN feature extraction is then used to extract road surface features such as cracks, potholes, and smoothness. The expression is:

[0050]

[0051] In the formula, z i This represents the value of the i-th point in the original data. It is the new value of the i-th point after bilateral filtering, z j p represents the original value of the j-th neighbor node. i p represents the spatial location of point i. j Represents the spatial position of point j, ||p i -p j || 2Let z represent the square of the Euclidean distance between points i and j in space. i -z j ) 2 Let represent the square of the numerical difference between points i and j. The standard deviation of spatial distance The standard deviation represents the numerical difference, and exp(·) represents the exponential function. Let W represent the neighborhood set of point i. i The normalization factor represents the unified format and normalization of key equipment operating parameters and environmental data in the real-time monitoring data, and completes time alignment. The spatiotemporal calibration technology and multimodal fusion model are used to achieve effective correlation between point data and surface data through the spatiotemporal calibration technology. The multimodal fusion model is based on the Transformer architecture and integrates the multimodal information of the three-dimensional terrain data, key equipment operating parameters and environmental data to generate a unified data feature vector.

[0052] Furthermore, the operation process of the digital twin module includes:

[0053] The digital twin model utilizes a point cloud generated by LiDAR to construct a continuous terrain function through surface fitting, expressed as:

[0054]

[0055] In the formula, Indicates all fitting coefficients α kl Find the set of values ​​that minimizes the objective function. This indicates that the summation is performed on all N sampling points. Represents the corresponding point (x) i ,y i The fitted value of ) Let represent the magnitude of the error, λ be the regularization coefficient, and ∫…dxdy represent the integral over the entire two-dimensional space. Let x represent the second derivative of the surface. Let represent the second derivative of the surface y. z represents the sum and square of all curvatures. i This represents the value of the i-th point in the original data. This represents the result of fitting the function at position (x, y) on a two-dimensional plane. This indicates that the index k is incremented from 0 to K. This indicates that index l is incremented from 0 to L, α kl Represents the fitting coefficient. Represents the k-th element with respect to x. spline basis functions Represents the l-th term with respect to y. Spline basis functions use a high-precision 3D terrain model constructed by lidar as a spatial reference, and equipment operating parameters and environmental data as dynamic inputs. By embedding a physical modeling engine and multi-objective optimization algorithm, they dynamically drive the virtual construction process and map and predict the trend of construction quality changes in real time, including but not limited to the response relationship between paving speed, number of compaction passes and compaction degree.

[0056] The digital twin model is used to conduct simulation and optimization evaluation of the combination of equipment operating parameters and environmental data, calculate the minimum paving time, the standard deviation of compaction degree, and the power consumption per unit area, and form a multi-objective optimization problem expression:

[0057]

[0058] In the formula, θ represents the set of parameters optimized in the digital twin model, and v p P(t) is the forward speed of the paver at time t, P(t) is the total power of all key construction equipment at time t, n(t) is the number of times the roller compacts the current area at time t, and Θ represents the feasible region of the parameters. The objective is to minimize these three objective functions. This indicates the penalty target for the speed of paving. Indicates the standard deviation of compaction. This represents the energy consumption cost per unit area. This indicates that the time integral covers the entire road construction period from time 0 to time T. Ω represents the reciprocal of the paver's forward velocity at time t, and Ω represents the two-dimensional spatial extent of the construction area. Ω …dxdy represents the area integral of the variable over the entire construction area Ω, ξ total (x,y,T) represents the compaction value measured at position x,y and time T. This indicates the average compaction degree of the area. This represents the square of the difference between the compaction degree at each point and the average value. This represents the summation of all M pieces of equipment at the road construction site, v k (t) represents the operating speed of the k-th device at time t, P k (t)·v k (t) is equivalent to the energy output per unit time multiplied by the processing rate, and the simulation result is output.

[0059] Furthermore, the operation procedure of the monitoring and early warning module includes:

[0060] When the monitoring results indicate quality risks including, but not limited to, insufficient compaction and abnormal paving thickness, the set intelligent early warning mechanism is automatically triggered. A risk discrimination function is defined using discrimination logic, all risk items are combined into a risk vector, and a comprehensive early warning intensity score is defined, expressed as:

[0061]

[0062] In the formula, S aet (x,y,t) represents the warning score at coordinates x,y and time t. This represents a weighted summation of K risk factors, γ k R represents the weighting coefficient for the k-th risk. k (x,y,t) represents the risk score value of the k-th risk factor at coordinates x,y and time t. Based on the digital twin model, multiple adjustment schemes are simulated, and the optimal scheme is selected and fed back to the external device system through a closed loop control command.

[0063] After the adjustment plan is implemented, the drone will conduct a follow-up inspection of the corrected area to collect inspection data.

[0064]

[0065] In the formula, X cek (x,y) represents the vector of road surface quality parameters obtained at the coordinate point (x,y) after on-site inspection by a drone. This represents the compaction degree measured after a re-check at the coordinate point (x, y). This represents the paving thickness measured after verification at coordinate point (x, y).

[0066] (x,y) represents the flatness index measured after verification at coordinate point (x,y), and the corrected validity function is calculated. The expression is:

[0067] η(x,y)=exp(-‖X cek (x,y)-X tgt ‖2)

[0068] In the formula, η(x,y) represents the corrected validity score at the two-dimensional coordinate point (x,y), with a value range of (0,1], and is a natural exponential function. tgt The vector represents the ideal index value of the expected target, and ||·|2 represents the Euclidean norm. The state of the digital twin model is updated based on the review data, and all construction process data is uniformly stored in the external data platform and data lake system.

[0069] Beneficial effects

[0070] Compared with known public technologies, the technical solution provided by this invention has the following beneficial effects:

[0071] In use, this invention breaks down the data fragmentation in the monitoring, analysis, and control stages of traditional construction monitoring by acquiring and fusing multi-source data, and constructing and applying digital twin models. It achieves fully automated response from data acquisition to process adjustment, reducing manual intervention costs by up to 70%. Through cross-validation of multimodal data, and combining visual inspection, equipment parameter monitoring, and simulation prediction results, the defect identification accuracy is increased to 95%, enhancing the control precision of the construction process, reducing the incidence of quality problems by up to 60%, and improving the accuracy of quality control. The pre-simulation function of the digital twin model can quickly evaluate different construction schemes, shortening process adjustment time by up to 80%. The dynamic monitoring of drones can promptly detect problems, reducing repetitive inspection workload by 50%, and improving overall construction efficiency by 30%, which is conducive to improving construction efficiency. Attached Figure Description

[0072] Figure 1 This is a flowchart of a construction process quality monitoring method based on multimodal data according to the present invention;

[0073] Figure 2 This is a system diagram of a construction process quality monitoring system based on multimodal data according to the present invention. Detailed Implementation

[0074] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0075] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0076] The present invention will now be described in further detail with reference to the accompanying drawings:

[0077] Example 1:

[0078] like Figure 1 As shown, this invention provides a construction process quality monitoring method based on multimodal data, including:

[0079] S1. The construction road surface area is perceived in all directions through a multimodal sensing system to obtain real-time monitoring data;

[0080] Furthermore, in step S1, the method for obtaining real-time monitoring data by using a multimodal sensing system to perform omnidirectional sensing of the construction pavement area is as follows:

[0081] The process involves a multimodal perception system to comprehensively perceive the construction pavement area. UAVs equipped with optical cameras, infrared thermal imagers, and lidar are used to acquire images, temperature fields, and 3D terrain data. Simultaneously, IoT monitoring units are deployed on key equipment, including but not limited to mixing plants, pavers, rollers, and transport vehicles, to collect their operating parameters. Environmental sensors are also deployed at the construction site to obtain environmental data such as temperature, humidity, and wind speed. The sampling points of the 3D terrain data, key equipment operating parameters, and environmental data are then aligned using a Gaussian kernel weighting function and a BeiDou positioning system timestamp. The expression is:

[0082]

[0083] In the formula, κ ain (a,b) represents the alignment similarity function, where the subscript ain indicates that this is a kernel function used for alignment, outputting a similarity value between 0 and 1 for a and b, ‖ab‖. 2 Let (a, b) represent the squared Euclidean distance between vectors a and b, and σ represent the standard deviation of the Gaussian kernel. 2 It is twice the square of the standard deviation, and exp(·) represents the exponent with the natural logarithm as the base, thus obtaining real-time monitoring data.

[0084] In this embodiment, the drone uses an optical camera to identify the smoothness of the road surface, an infrared thermal imager to monitor whether the asphalt paving temperature meets the standard, a lidar to construct a three-dimensional model of the road surface, and an IoT unit on the paver to transmit parameters such as paving speed and vibration frequency in real time. Environmental sensors simultaneously collect on-site wind speed, and the Beidou system marks all data with precise time. After alignment using a Gaussian kernel function, construction managers can grasp the correlation data of paving temperature, equipment operating status, and environmental conditions in real time. Through real-time monitoring of multi-dimensional data, construction process parameters can be accurately controlled, avoiding quality problems such as road surface cracking and rutting caused by human error. Real-time data alignment and analysis can provide early warning of equipment failures or process deviations, reduce downtime for rectification, form a standardized construction data chain, and provide quantitative reference for subsequent projects.

[0085] S2. The real-time monitoring data is preprocessed and time-aligned. Simultaneously, spatiotemporal calibration technology and multimodal fusion model are used to generate data feature vectors.

[0086] Further, in step S2, the real-time monitoring data is preprocessed and time-aligned. Simultaneously, the data feature vector is generated using spatiotemporal calibration technology and a multimodal fusion model.

[0087] The real-time monitoring data is preprocessed and time-aligned. A bilateral filter is introduced to denoise and enhance the 3D terrain data within the real-time monitoring data. CNN feature extraction is then used to extract road surface features such as cracks, potholes, and smoothness. The expression is:

[0088]

[0089] In the formula, z i This represents the value of the i-th point in the original data. It is the new value of the i-th point after bilateral filtering, z j p represents the original value of the j-th neighbor node. i p represents the spatial location of point i. j Represents the spatial position of point j, ||p i -p j || 2 Let z represent the square of the Euclidean distance between points i and j in space. i -z j ) 2 Let represent the square of the numerical difference between points i and j. The standard deviation of spatial distance The standard deviation represents the numerical difference, and exp(·) represents the exponential function. Let W represent the neighborhood set of point i. iThe normalization factor represents the unified format and normalization of key equipment operating parameters and environmental data in the real-time monitoring data, and completes time alignment. The spatiotemporal calibration technology and multimodal fusion model are used to achieve effective correlation between point data and surface data through the spatiotemporal calibration technology. The multimodal fusion model is based on the Transformer architecture and integrates the multimodal information of the three-dimensional terrain data, key equipment operating parameters and environmental data to generate a unified data feature vector.

[0090] In this embodiment, the 3D point cloud data of the road surface collected by the UAV contains noise caused by changes in lighting. After processing with a bilateral filter, it clearly shows that there are three minor cracks within a 10-meter range of a certain road section. At the same time, the paving speed and vibration frequency data transmitted by the paver's IoT unit, and the humidity data collected by the environmental sensor, after being formatted and time-aligned, are accurately matched with the processed 3D terrain data. Through spatiotemporal calibration technology, this invention associates the paver's operating parameters at a specific location with the terrain features of the corresponding road area. Then, using the Transformer fusion model, all data are integrated to generate feature vectors, ultimately intuitively demonstrating the causal relationship that high humidity in a certain road section leads to a decrease in the paver's vibration effect, causing local unevenness of the road surface. This effectively removes data noise, accurately identifies road defects, avoids the problem of missed detection by manual inspection, and improves the detection accuracy to over 95%.

[0091] The Transformer model breaks down data barriers and enables deep fusion of multi-source heterogeneous data, which is beneficial for predicting asphalt paving quality problems caused by abnormal humidity.

[0092] S3. Construct a digital twin model of the construction process based on the data feature vectors, perform simulation and optimization evaluation, and output the simulation results;

[0093] Furthermore, in step S3, the method for constructing a digital twin model of the construction process based on the data feature vectors, performing simulation and optimization evaluation, and outputting the simulation results is as follows:

[0094] The digital twin model utilizes a point cloud generated by LiDAR to construct a continuous terrain function through surface fitting, expressed as:

[0095]

[0096] In the formula, Indicates all fitting coefficients α kl Find the set of values ​​that minimizes the objective function. This indicates that the summation is performed on all N sampling points. Represents the corresponding point (x) i ,y iThe fitted value of ) Let represent the magnitude of the error, λ be the regularization coefficient, and ∫…dxdy represent the integral over the entire two-dimensional space. Let x represent the second derivative of the surface. Let represent the second derivative of the surface y. z represents the sum and square of all curvatures. i This represents the value of the i-th point in the original data. This represents the result of fitting the function at position (x, y) on a two-dimensional plane. This indicates that the index k is incremented from 0 to K. This indicates that index l is incremented from 0 to L, α kl Represents the fitting coefficient. Represents the k-th element with respect to x. spline basis functions Represents the l-th term with respect to y. Spline basis functions use a high-precision 3D terrain model constructed by lidar as a spatial reference, and equipment operating parameters and environmental data as dynamic inputs. By embedding a physical modeling engine and multi-objective optimization algorithm, they dynamically drive the virtual construction process and map and predict the trend of construction quality changes in real time, including but not limited to the response relationship between paving speed, number of compaction passes and compaction degree.

[0097] Furthermore, in step S3, the method for constructing a digital twin model of the construction process based on the data feature vectors, performing simulation and optimization evaluation, and outputting the simulation results is as follows:

[0098] The digital twin model is used to conduct simulation and optimization evaluation of the combination of equipment operating parameters and environmental data, calculate the minimum paving time, the standard deviation of compaction degree, and the power consumption per unit area, and form a multi-objective optimization problem expression:

[0099]

[0100] In the formula, θ represents the set of parameters optimized in the digital twin model, and v p P(t) is the forward speed of the paver at time t, P(t) is the total power of all key construction equipment at time t, n(t) is the number of times the roller compacts the current area at time t, and Θ represents the feasible region of the parameters. The objective is to minimize these three objective functions. This indicates the penalty target for the speed of paving. Indicates the standard deviation of compaction. This represents the energy consumption cost per unit area. This indicates that the time integral covers the entire road construction period from time 0 to time T. Ω represents the reciprocal of the paver's forward velocity at time t, and Ω represents the two-dimensional spatial extent of the construction area. Ω …dxdy represents the area integral of the variable over the entire construction area Ω, ξ total (x,y,T) represents the compaction value measured at position x,y and time T. This indicates the average compaction degree of the area. This represents the square of the difference between the compaction degree at each point and the average value. This represents the summation of all M pieces of equipment at the road construction site, v k (t) represents the operating speed of the k-th device at time t, P k (t)·v k (t) is equivalent to the energy output per unit time multiplied by the processing rate, and the simulation result is output.

[0101] In this embodiment, after obtaining the processed data feature vectors, a digital twin model of the construction process is constructed using these data. Through simulation using the digital twin model, it is found that under the conditions of an ambient temperature of 25°C and a humidity of 60%, the combination of the current paver speed and the number of compaction passes of the roller will lead to an excessively large standard deviation of compaction in some road sections, affecting the pavement quality. Therefore, a multi-objective optimization algorithm is used to adjust and simulate the parameters, ultimately determining that the paver should move forward at a suitable speed, combined with a specific number of compaction passes of the roller. This ensures compaction while reducing power consumption per unit area by 10% and shortening paving time by 15%. The simulation results provide accurate parameter guidance for actual construction. Through digital twin model simulation and multi-objective optimization, the optimal combination of construction parameters can be accurately found, avoiding quality defects and resource waste caused by unreasonable parameter settings.

[0102] S4. Based on the real-time monitoring data and simulation results, perform comparative analysis, dynamically monitor road surface quality indicators, output monitoring results, and set up an intelligent early warning mechanism;

[0103] Furthermore, in step S4, the method for comparing and analyzing the real-time monitoring data with the simulation results, dynamically monitoring road surface quality indicators, outputting monitoring results, and setting up an intelligent early warning mechanism is as follows:

[0104] When the monitoring results indicate quality risks including, but not limited to, insufficient compaction and abnormal paving thickness, the set intelligent early warning mechanism is automatically triggered. A risk discrimination function is defined using discrimination logic, all risk items are combined into a risk vector, and a comprehensive early warning intensity score is defined, expressed as:

[0105]

[0106] In the formula, S aet(x,y,t) represents the warning score at coordinates x,y and time t. This represents a weighted summation of K risk factors, γ k R represents the weighting coefficient for the k-th risk. k (x,y,t) represents the risk score value of the k-th risk factor at coordinates x,y and time t. Based on the digital twin model, multiple adjustment schemes are simulated, and the optimal scheme is selected and fed back to the external device system through a closed loop control command.

[0107] Furthermore, in step S4, the method for comparing and analyzing the real-time monitoring data with the simulation results, dynamically monitoring road surface quality indicators, outputting monitoring results, and setting up an intelligent early warning mechanism is as follows:

[0108] After the adjustment plan is implemented, the drone will conduct a follow-up inspection of the corrected area to collect inspection data.

[0109]

[0110] In the formula, X cek (x,y) represents the vector of road surface quality parameters obtained at the coordinate point (x,y) after on-site inspection by a drone. This represents the compaction degree measured after a re-check at the coordinate point (x, y). This represents the paving thickness measured after verification at coordinate point (x, y).

[0111] (x,y) represents the flatness index measured after verification at coordinate point (x,y), and the corrected validity function is calculated. The expression is:

[0112] η(x,y)=exp(-‖X cek (x,y)-X tgt ‖2)

[0113] In the formula, η(x,y) represents the corrected validity score at the two-dimensional coordinate point (x,y), with a value range of (0,1], and is a natural exponential function. tgt The vector represents the ideal index value of the expected target, and ||·|2 represents the Euclidean norm. The state of the digital twin model is updated based on the review data, and all construction process data is uniformly stored in the external data platform and data lake system.

[0114] In this embodiment, after simulating the construction process, real-time monitoring data is compared and analyzed with the simulation results to dynamically monitor pavement quality indicators. An intelligent early warning mechanism ensures construction quality. During pavement construction, real-time monitoring data shows that the compaction degree of a certain section is only 90%, lower than the 95% standard expected in the simulation results, triggering an intelligent early warning. Through risk discrimination function analysis, it is determined that the risk weight of insufficient compaction is high. Three adjustment schemes are simulated using a digital twin model: increasing the number of roller passes, reducing the paving speed, and increasing the roller vibration frequency. After comparison and evaluation, the scheme of increasing roller passes by 2 times is selected and an instruction is issued. After the adjustment scheme is executed, a drone re-inspects the section, collecting data such as compaction degree, paving thickness, and smoothness. The calculated correction effectiveness score is 0.85, indicating that the adjustment scheme is effective and helps to identify quality risks in a timely manner, preventing the problem from escalating. Through automated monitoring, early warning, and feedback processes, manual intervention is reduced, improving the intelligence and precision of construction management.

[0115] Example 2:

[0116] like Figure 2 As shown, Example 2 provides a construction process quality monitoring system based on multimodal data, including:

[0117] The data acquisition module uses a multimodal sensing system to perceive the construction road surface area from all angles and obtain real-time monitoring data.

[0118] The data fusion module preprocesses the real-time monitoring data and performs time alignment, while using spatiotemporal calibration technology and a multimodal fusion model to generate data feature vectors.

[0119] The digital twin module constructs a digital twin model of the construction process based on the data feature vectors, performs simulation and optimization evaluation, and outputs simulation results;

[0120] The monitoring and early warning module compares and analyzes the real-time monitoring data with the simulation results, dynamically monitors road surface quality indicators, outputs monitoring results, and sets up an intelligent early warning mechanism.

[0121] Furthermore, the operation process of the data acquisition module includes:

[0122] The process involves a multimodal perception system to comprehensively perceive the construction pavement area. UAVs equipped with optical cameras, infrared thermal imagers, and lidar are used to acquire images, temperature fields, and 3D terrain data. Simultaneously, IoT monitoring units are deployed on key equipment, including but not limited to mixing plants, pavers, rollers, and transport vehicles, to collect their operating parameters. Environmental sensors are also deployed at the construction site to obtain environmental data such as temperature, humidity, and wind speed. The sampling points of the 3D terrain data, key equipment operating parameters, and environmental data are then aligned using a Gaussian kernel weighting function and a BeiDou positioning system timestamp. The expression is:

[0123]

[0124] In the formula, κ ain (a,b) represents the alignment similarity function, where the subscript ain indicates that this is a kernel function used for alignment, outputting a similarity value between 0 and 1 for a and b, ‖ab‖. 2 Let (a, b) represent the squared Euclidean distance between vectors a and b, and σ represent the standard deviation of the Gaussian kernel. 2 It is twice the square of the standard deviation, and exp(·) represents the exponent with the natural logarithm as the base, thus obtaining real-time monitoring data;

[0125] Furthermore, the operation process of the data fusion module includes:

[0126] The real-time monitoring data is preprocessed and time-aligned. A bilateral filter is introduced to denoise and enhance the 3D terrain data within the real-time monitoring data. CNN feature extraction is then used to extract road surface features such as cracks, potholes, and smoothness. The expression is:

[0127]

[0128] In the formula, z i This represents the value of the i-th point in the original data. It is the new value of the i-th point after bilateral filtering, z j p represents the original value of the j-th neighbor node. i p represents the spatial location of point i. j Represents the spatial position of point j, ||p i -p j || 2 Let z represent the square of the Euclidean distance between points i and j in space. i -z j ) 2 Let represent the square of the numerical difference between points i and j. The standard deviation of spatial distance The standard deviation represents the numerical difference, and exp(·) represents the exponential function. Let W represent the neighborhood set of point i. i The normalization factor represents the unified format and normalization of key equipment operating parameters and environmental data in the real-time monitoring data, and completes time alignment. The spatiotemporal calibration technology and multimodal fusion model are used to achieve effective correlation between point data and surface data through the spatiotemporal calibration technology. The multimodal fusion model is based on the Transformer architecture and integrates the multimodal information of the three-dimensional terrain data, key equipment operating parameters and environmental data to generate a unified data feature vector.

[0129] Furthermore, the operation process of the digital twin module includes:

[0130] The digital twin model utilizes a point cloud generated by LiDAR to construct a continuous terrain function through surface fitting, expressed as:

[0131]

[0132] In the formula, Indicates all fitting coefficients α kl Find the set of values ​​that minimizes the objective function. This indicates that the summation is performed on all N sampling points. Represents the corresponding point (x) i ,y i The fitted value of ) Let represent the magnitude of the error, λ be the regularization coefficient, and ∫…dxdy represent the integral over the entire two-dimensional space. Let x represent the second derivative of the surface. Let represent the second derivative of the surface y. z represents the sum and square of all curvatures. i This represents the value of the i-th point in the original data. This represents the result of fitting the function at position (x, y) on a two-dimensional plane. This indicates that the index k is incremented from 0 to K. This indicates that index l is incremented from 0 to L, α kl Represents the fitting coefficient. Represents the k-th element with respect to x. spline basis functions Represents the l-th term with respect to y. Spline basis functions use a high-precision 3D terrain model constructed by lidar as a spatial reference, and equipment operating parameters and environmental data as dynamic inputs. By embedding a physical modeling engine and multi-objective optimization algorithm, they dynamically drive the virtual construction process and map and predict the trend of construction quality changes in real time, including but not limited to the response relationship between paving speed, number of compaction passes and compaction degree.

[0133] The digital twin model is used to conduct simulation and optimization evaluation of the combination of equipment operating parameters and environmental data, calculate the minimum paving time, the standard deviation of compaction degree, and the power consumption per unit area, and form a multi-objective optimization problem expression:

[0134]

[0135] In the formula, θ represents the set of parameters optimized in the digital twin model, and v p P(t) is the forward speed of the paver at time t, P(t) is the total power of all key construction equipment at time t, n(t) is the number of times the roller compacts the current area at time t, and Θ represents the feasible region of the parameters. The objective is to minimize these three objective functions. This indicates the penalty target for the speed of paving. Indicates the standard deviation of compaction. This represents the energy consumption cost per unit area. This indicates that the time integral covers the entire road construction period from time 0 to time T. Ω represents the reciprocal of the paver's forward velocity at time t, and Ω represents the two-dimensional spatial extent of the construction area. Ω …dxdy represents the area integral of the variable over the entire construction area Ω, ξ total (x,y,T) represents the compaction value measured at position x,y and time T. This indicates the average compaction degree of the area. This represents the square of the difference between the compaction degree at each point and the average value. This represents the summation of all M pieces of equipment at the road construction site, v k (t) represents the operating speed of the k-th device at time t, P k (t)·v k (t) is equivalent to the energy output per unit time multiplied by the processing rate, and the simulation result is output.

[0136] Furthermore, the operation procedure of the monitoring and early warning module includes:

[0137] When the monitoring results indicate quality risks including, but not limited to, insufficient compaction and abnormal paving thickness, the set intelligent early warning mechanism is automatically triggered. A risk discrimination function is defined using discrimination logic, all risk items are combined into a risk vector, and a comprehensive early warning intensity score is defined, expressed as:

[0138]

[0139] In the formula, S aet (x,y,t) represents the warning score at coordinates x,y and time t. This represents a weighted summation of K risk factors, γ k R represents the weighting coefficient for the k-th risk. k (x,y,t) represents the risk score value of the k-th risk factor at coordinates x,y and time t. Based on the digital twin model, multiple adjustment schemes are simulated, and the optimal scheme is selected and fed back to the external device system through a closed loop control command.

[0140] After the adjustment plan is implemented, the drone will conduct a follow-up inspection of the corrected area to collect inspection data.

[0141]

[0142] In the formula, X cek (x,y) represents the vector of road surface quality parameters obtained at the coordinate point (x,y) after on-site inspection by a drone. This represents the compaction degree measured after a re-check at the coordinate point (x, y). This represents the paving thickness measured after a re-inspection at coordinate point (x,y), and ^(x,y) represents the smoothness index measured after a re-inspection at coordinate point (x,y). A correction validity function is calculated, expressed as:

[0143] η(x,y)=exp(-‖X cek (x,y)-X tgt ‖2)

[0144] In the formula, η(x,y) represents the corrected validity score at the two-dimensional coordinate point (x,y), with a value range of (0,1], and is a natural exponential function. tgt The vector represents the ideal index value of the expected target, and ||·|2 represents the Euclidean norm. The state of the digital twin model is updated based on the review data, and all construction process data is uniformly stored in the external data platform and data lake system.

[0145] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions will not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A construction process quality monitoring method based on multimodal data, characterized in that, include: S1. The construction road surface area is perceived in all directions through a multimodal sensing system to obtain real-time monitoring data; S2. The real-time monitoring data is preprocessed and time-aligned. Simultaneously, spatiotemporal calibration technology and multimodal fusion model are used to generate data feature vectors. S3. Construct a digital twin model of the construction process based on the data feature vectors, perform simulation and optimization evaluation, and output the simulation results; S4. Based on the real-time monitoring data and simulation results, perform comparative analysis, dynamically monitor road surface quality indicators, output monitoring results, and set up an intelligent early warning mechanism.

2. The construction process quality monitoring method based on multimodal data according to claim 1, characterized in that, In step S1, the method for obtaining real-time monitoring data by using a multimodal sensing system to perform all-round sensing of the construction pavement area is as follows: The process involves a multimodal perception system to comprehensively perceive the construction pavement area. UAVs equipped with optical cameras, infrared thermal imagers, and lidar are used to acquire images, temperature fields, and 3D terrain data. Simultaneously, IoT monitoring units are deployed on key equipment, including but not limited to mixing plants, pavers, rollers, and transport vehicles, to collect their operating parameters. Environmental sensors are also deployed at the construction site to obtain environmental data such as temperature, humidity, and wind speed. The sampling points of the 3D terrain data, key equipment operating parameters, and environmental data are then aligned using a Gaussian kernel weighting function and a BeiDou positioning system timestamp. The expression is: In the formula, κ ain (a,b) represents the alignment similarity function, where the subscript ain indicates that this is a kernel function used for alignment, outputting a similarity value between 0 and 1 for a and b, ‖ab‖. 2 Let (a, b) represent the squared Euclidean distance between vectors a and b, and σ represent the standard deviation of the Gaussian kernel. 2 It is twice the square of the standard deviation, and exp(·) represents the exponent with the natural logarithm as the base, thus obtaining real-time monitoring data.

3. The construction process quality monitoring method based on multimodal data according to claim 2, characterized in that, In step S2, the real-time monitoring data is preprocessed and time-aligned. Simultaneously, spatiotemporal calibration technology and a multimodal fusion model are used to generate data feature vectors. The real-time monitoring data is preprocessed and time-aligned. A bilateral filter is introduced to denoise and enhance the 3D terrain data within the real-time monitoring data. CNN feature extraction is then used to extract road surface features such as cracks, potholes, and smoothness. The expression is: In the formula, z i This represents the value of the i-th point in the original data. It is the new value of the i-th point after bilateral filtering, z j p represents the original value of the j-th neighbor point. i p represents the spatial location of point i. j Represents the spatial position of point j, ||p i -p j || 2 Let z represent the square of the Euclidean distance between points i and j in space. i -z j ) 2 Let represent the square of the numerical difference between points i and j. The standard deviation of spatial distance The standard deviation represents the numerical difference, and exp(·) represents the exponential function. Let W represent the neighborhood set of point i. i The normalization factor represents the unified format and normalization of key equipment operating parameters and environmental data in the real-time monitoring data, and completes time alignment. The spatiotemporal calibration technology and multimodal fusion model are used to achieve effective correlation between point data and surface data through the spatiotemporal calibration technology. The multimodal fusion model is based on the Transformer architecture and integrates the multimodal information of the three-dimensional terrain data, key equipment operating parameters and environmental data to generate a unified data feature vector.

4. The construction process quality monitoring method based on multimodal data according to claim 3, characterized in that, In step S3, the method for constructing a digital twin model of the construction process based on the data feature vectors, performing simulation and optimization evaluation, and outputting the simulation results is as follows: The digital twin model utilizes a point cloud generated by LiDAR to construct a continuous terrain function through surface fitting, expressed as: In the formula, Indicates all fitting coefficients α kl Find the set of values ​​that minimizes the objective function. This indicates that the summation is performed on all N sampling points. Represents the corresponding point (x) i ,y i The fitted value of ) Let represent the magnitude of the error, λ be the regularization coefficient, and ∫…dxdy represent the integral over the entire two-dimensional space. Let x represent the second derivative of the surface. Let represent the second derivative of the surface y. z represents the sum and square of all curvatures. i This represents the value of the i-th point in the original data. This represents the result of fitting the function at position (x, y) on a two-dimensional plane. This indicates that the index k is incremented from 0 to K. This indicates that index l is incremented from 0 to L, α kl Represents the fitting coefficient. Represents the k-th element with respect to x. spline basis functions Represents the l-th term with respect to y. Spline basis functions use a high-precision 3D terrain model constructed by lidar as a spatial reference, and equipment operating parameters and environmental data as dynamic inputs. By embedding a physical modeling engine and multi-objective optimization algorithm, they dynamically drive the virtual construction process and map and predict the trend of construction quality changes in real time, including but not limited to the response relationship between paving speed, number of compaction passes and compaction degree.

5. The construction process quality monitoring method based on multimodal data according to claim 4, characterized in that, In step S3, the method for constructing a digital twin model of the construction process based on the data feature vectors, performing simulation and optimization evaluation, and outputting the simulation results is as follows: The digital twin model is used to conduct simulation and optimization evaluation of the combination of equipment operating parameters and environmental data, calculate the minimum paving time, the standard deviation of compaction degree, and the power consumption per unit area, and form a multi-objective optimization problem expression: In the formula, θ represents the set of parameters optimized in the digital twin model, and v p P(t) is the forward speed of the paver at time t, P(t) is the total power of all key construction equipment at time t, n(t) is the number of times the roller compacts the current area at time t, and Θ represents the feasible region of the parameters. The objective is to minimize these three objective functions. This indicates the penalty target for the speed of paving. Indicates the standard deviation of compaction. This represents the energy consumption cost per unit area. This indicates that the time integral covers the entire road construction period from time 0 to time T. Ω represents the reciprocal of the paver's forward velocity at time t, and Ω represents the two-dimensional spatial extent of the construction area. Ω …dxdy represents the area integral of the variable over the entire construction area Ω, ξ total (x,y,T) represents the compaction value measured at position x,y and time T. This indicates the average compaction degree of the area. This represents the square of the difference between the compaction degree at each point and the average value. This represents the summation of all M pieces of equipment at the road construction site, v k (t) represents the operating speed of the k-th device at time t, P k (t)·v k (t) is equivalent to the energy output per unit time multiplied by the processing rate, and the simulation result is output.

6. The construction process quality monitoring method based on multimodal data according to claim 5, characterized in that, In step S4, the method for comparing and analyzing the real-time monitoring data with the simulation results, dynamically monitoring road surface quality indicators, outputting monitoring results, and setting up an intelligent early warning mechanism is as follows: When the monitoring results indicate quality risks including, but not limited to, insufficient compaction and abnormal paving thickness, the set intelligent early warning mechanism is automatically triggered. A risk discrimination function is defined using discrimination logic, all risk items are combined into a risk vector, and a comprehensive early warning intensity score is defined, expressed as: In the formula, S aet (x,y,t) represents the warning score at coordinates x,y and time t. This represents a weighted summation of K risk factors, γ k R represents the weighting coefficient for the k-th risk. k (x,y,t) represents the risk score value of the k-th risk factor at coordinates x,y and time t. Based on the digital twin model, multiple adjustment schemes are simulated, and the optimal scheme is selected and fed back to the external device system through a closed loop control command.

7. The construction process quality monitoring method based on multimodal data according to claim 6, characterized in that, In step S4, the method for comparing and analyzing the real-time monitoring data with the simulation results, dynamically monitoring road surface quality indicators, outputting monitoring results, and setting up an intelligent early warning mechanism is as follows: After the adjustment plan is completed, the drone will re-inspect the corrected area to collect re-inspection data: In the formula, X cek (x,y) represents the vector of road surface quality parameters obtained at the coordinate point (x,y) after on-site inspection by a drone. This represents the compaction degree measured after a re-check at the coordinate point (x, y). This represents the paving thickness measured after a re-inspection at coordinate point (x,y), and ^(x,y) represents the smoothness index measured after a re-inspection at coordinate point (x,y). A correction validity function is calculated, expressed as: η(x,y)=exp(-‖X cek (x,y)-X tgt ‖2) In the formula, η(x,y) represents the corrected validity score at the two-dimensional coordinate point (x,y), with a value range of (0,1], and is a natural exponential function. tgt The vector represents the ideal index value of the expected target, and ||·|2 represents the Euclidean norm. The state of the digital twin model is updated based on the review data, and all construction process data is uniformly stored in the external data platform and data lake system.

8. A construction process quality monitoring system based on multimodal data, based on the construction process quality monitoring method based on multimodal data according to any one of claims 1-7, characterized in that, include: The data acquisition module uses a multimodal sensing system to perceive the construction road surface area from all angles and obtain real-time monitoring data. The data fusion module preprocesses the real-time monitoring data and performs time alignment, while using spatiotemporal calibration technology and a multimodal fusion model to generate data feature vectors. The digital twin module constructs a digital twin model of the construction process based on the data feature vectors, performs simulation and optimization evaluation, and outputs simulation results; The monitoring and early warning module compares and analyzes the real-time monitoring data with the simulation results, dynamically monitors road surface quality indicators, outputs monitoring results, and sets up an intelligent early warning mechanism.

9. A construction process quality monitoring system based on multimodal data according to claim 8, characterized in that, The operation process of the data acquisition module includes: The process involves a multimodal perception system to comprehensively perceive the construction pavement area. UAVs equipped with optical cameras, infrared thermal imagers, and lidar are used to acquire images, temperature fields, and 3D terrain data. Simultaneously, IoT monitoring units are deployed on key equipment, including but not limited to mixing plants, pavers, rollers, and transport vehicles, to collect their operating parameters. Environmental sensors are also deployed at the construction site to obtain environmental data such as temperature, humidity, and wind speed. The sampling points of the 3D terrain data, key equipment operating parameters, and environmental data are then aligned using a Gaussian kernel weighting function and a BeiDou positioning system timestamp. The expression is: In the formula, κ ain (a,b) represents the alignment similarity function, where the subscript ain indicates that this is a kernel function used for alignment, outputting a similarity value between 0 and 1 for a and b, ‖ab‖. 2 Let (a, b) represent the squared Euclidean distance between vectors a and b, and σ represent the standard deviation of the Gaussian kernel. 2 It is twice the square of the standard deviation, and exp(·) represents the exponent with the natural logarithm as the base, thus obtaining real-time monitoring data; The operation process of the data fusion module includes: The real-time monitoring data is preprocessed and time-aligned. A bilateral filter is introduced to denoise and enhance the 3D terrain data within the real-time monitoring data. CNN feature extraction is then used to extract road surface features such as cracks, potholes, and smoothness. The expression is: In the formula, z i This represents the value of the i-th point in the original data. It is the new value of the i-th point after bilateral filtering, z j p represents the original value of the j-th neighbor point. i p represents the spatial location of point i. j Represents the spatial position of point j, ||p i -p j || 2 Let z represent the square of the Euclidean distance between points i and j in space. i -z j ) 2 Let represent the square of the numerical difference between points i and j. The standard deviation of spatial distance The standard deviation represents the numerical difference, and exp(·) represents the exponential function. Let W represent the neighborhood set of point i. i The normalization factor represents the unified format and normalization of key equipment operating parameters and environmental data in the real-time monitoring data, and completes time alignment. The spatiotemporal calibration technology and multimodal fusion model are used to achieve effective correlation between point data and surface data through the spatiotemporal calibration technology. The multimodal fusion model is based on the Transformer architecture and integrates the multimodal information of the three-dimensional terrain data, key equipment operating parameters and environmental data to generate a unified data feature vector.

10. A construction process quality monitoring system based on multimodal data according to claim 9, characterized in that, The operation process of the digital twin module includes: The digital twin model utilizes a point cloud generated by LiDAR to construct a continuous terrain function through surface fitting, expressed as: In the formula, Indicates all fitting coefficients α kl Find the set of values ​​that minimizes the objective function. This indicates that the summation is performed on all N sampling points. Represents the corresponding point (x) i ,y i The fitted value of ) Let represent the magnitude of the error, λ be the regularization coefficient, and ∫…dxdy represent the integral over the entire two-dimensional space. Let x represent the second derivative of the surface. Let represent the second derivative of the surface y. z represents the sum and square of all curvatures. i This represents the value of the i-th point in the original data. This represents the result of fitting the function at position (x, y) on a two-dimensional plane. This indicates that the index k is incremented from 0 to K. This indicates that index l is incremented from 0 to L, α kl Represents the fitting coefficient. Represents the k-th element with respect to x. spline basis functions Represents the l-th term with respect to y. Spline basis functions use a high-precision 3D terrain model constructed by lidar as a spatial reference, and equipment operating parameters and environmental data as dynamic inputs. By embedding a physical modeling engine and multi-objective optimization algorithm, they dynamically drive the virtual construction process and map and predict the trend of construction quality changes in real time, including but not limited to the response relationship between paving speed, number of compaction passes and compaction degree. The digital twin model is used to conduct simulation and optimization evaluation of the combination of equipment operating parameters and environmental data, calculate the minimum paving time, the standard deviation of compaction degree, and the power consumption per unit area, and form a multi-objective optimization problem expression: In the formula, θ represents the set of parameters optimized in the digital twin model, and v p P(t) is the forward speed of the paver at time t, P(t) is the total power of all key construction equipment at time t, n(t) is the number of times the roller compacts the current area at time t, and Θ represents the feasible region of the parameters. The objective is to minimize these three objective functions. This indicates the penalty target for the speed of paving. Indicates the standard deviation of compaction. This represents the energy consumption cost per unit area. This indicates that the time integral covers the entire road construction period from time 0 to time T. Ω represents the reciprocal of the paver's forward velocity at time t, and Ω represents the two-dimensional spatial extent of the construction area. Ω …dxdy represents the area integral of the variable over the entire construction area Ω, ξ total (x,y,T) represents the compaction value measured at position x,y and time T. This indicates the average compaction degree of the area. This represents the square of the difference between the compaction degree at each point and the average value. This represents the summation of all M pieces of equipment at the road construction site, v k (t) represents the operating speed of the k-th device at time t, P k (t)·v k (t) is equivalent to the energy output per unit time multiplied by the processing rate, and the simulation result is output. The operation process of the monitoring and early warning module includes: When the monitoring results indicate quality risks including, but not limited to, insufficient compaction and abnormal paving thickness, the set intelligent early warning mechanism is automatically triggered. A risk discrimination function is defined using discrimination logic, all risk items are combined into a risk vector, and a comprehensive early warning intensity score is defined, expressed as: In the formula, S aet (x,y,t) represents the warning score at coordinates x,y and time t. This represents a weighted summation of K risk factors, γ k R represents the weighting coefficient for the k-th risk. k (x,y,t) represents the risk score value of the k-th risk factor at coordinates x,y and time t. Based on the digital twin model, multiple adjustment schemes are simulated, and the optimal scheme is selected and fed back to the external device system through a closed loop control command. After the adjustment plan is completed, the drone will re-inspect the corrected area to collect re-inspection data: In the formula, X cek (x,y) represents the vector of road surface quality parameters obtained at the coordinate point (x,y) after on-site inspection by a drone. This represents the compaction degree measured after a re-check at the coordinate point (x, y). This represents the paving thickness measured after a re-inspection at coordinate point (x,y), and ^(x,y) represents the smoothness index measured after a re-inspection at coordinate point (x,y). A correction validity function is calculated, expressed as: η(x,y)=exp(-‖X cek (x,y)-X tgt ‖2) In the formula, η(x,y) represents the corrected validity score at the two-dimensional coordinate point (x,y), with a value range of (0,1], and is a natural exponential function. tgt The vector represents the ideal index value of the expected target, and ||·|2 represents the Euclidean norm. The state of the digital twin model is updated based on the review data, and all construction process data is uniformly stored in the external data platform and data lake system.

Images