Method and system for monitoring asphalt paving uniformity in rainy days based on multi-source sensing data

Abstract

The invention discloses a rainy day asphalt paving uniformity monitoring method and system based on multi-source sensing data, and belongs to the technical field of road engineering construction, and the method comprises the steps: S1, obtaining an original infrared thermogram, an original radar point cloud and environment weather data in a rainy day asphalt paving process, and constructing an original infrared thermogram sequence and an original radar point cloud sequence; s2, performing multi-modal data synchronization and space-time tensor construction to obtain a multi-modal space-time tensor sequence; s3, performing depression detection on the multi-modal space-time tensor sequence to obtain a depression attention map; s4, based on the depression attention map, space-time network heat map reconstruction based on physical constraint is carried out, and a reconstructed heat map sequence is obtained; s5, on the basis of real-time weather data, pavement materials and structural parameters, performing compaction window period deduction on the reconstructed heat map sequence to obtain a future temperature prediction map; and S6, generating a compaction guidance map based on the current reconstruction temperature field and the future temperature prediction map to obtain the compaction guidance map.

Description

Technical Field

[0001] This invention belongs to the field of road engineering construction technology, specifically relating to a method and system for monitoring the uniformity of asphalt paving in rainy weather based on multi-source sensor data. Background Technology

[0002] In the construction of asphalt pavements, paving uniformity, especially the uniformity of density determined by the final compaction stage, is always a core indicator for measuring overall construction quality. The compaction effect of asphalt mixtures is highly dependent on their temperature state. Only when compaction is carried out within an appropriate temperature window can the mixture possess sufficient plasticity, allowing aggregate particles to fully interlock and rearrange, ultimately forming a stable and dense pavement structure. This temperature window typically lies within a specific time period after paving; beyond this period, asphalt viscosity increases and compactability significantly decreases. However, in actual construction, especially during paving operations under adverse weather conditions such as rain, effectively monitoring the temperature field of the asphalt paving layer and maintaining its uniformity presents a significant challenge. Rainwater not only rapidly removes heat from the asphalt surface but also causes localized water accumulation due to uneven paving surfaces or poor drainage, leading to a rapid and highly uneven temperature drop. This uneven cooling significantly shortens the effective compaction time window. Without a real-time, accurate temperature monitoring and feedback system to guide roller operations, insufficient compaction in localized areas is highly likely. These weakly compacted areas are prone to serious quality defects such as early loosening, aggregate spalling, potholes and water damage under the combined effects of traffic loads and water erosion, which seriously affect the service life of the road surface and driving safety.

[0003] To address the quality monitoring needs of asphalt paving, existing technologies have proposed several solutions. One mainstream method uses an infrared thermal imager to scan the entire asphalt surface temperature immediately after paving, indirectly assessing the uniformity of the mixture distribution by identifying low-temperature areas. However, this technology has a fundamental limitation in rainy environments: rainwater forms flowing water films or stagnant droplets on the asphalt surface. In this case, the infrared thermal imager actually detects the surface temperature of the water, not the true internal temperature of the asphalt mixture. Furthermore, raindrops impacting the paved surface introduce random and intense thermal noise, severely interfering with the true temperature field distribution, leading to distorted measurement data and failing to effectively guide the compaction process.

[0004] Another widely used technology is smart compaction, which uses sensors (such as accelerometers or deformation measuring instruments) mounted on the roller's steel drum to calculate the stiffness or modulus of the asphalt material by analyzing the feedback vibration response, thereby assessing the degree of compaction. However, this method is essentially a reactive, post-hoc monitoring approach—compaction quality data can only be obtained after the roller has passed a section of road. Once insufficient compaction is detected, the asphalt mixture may have already cooled below the effective compaction temperature, making effective secondary compaction difficult. This lag is particularly pronounced under conditions of sudden temperature drops during rain, making real-time process control virtually impossible.

[0005] Therefore, developing and constructing a technical solution that can reliably and in real-time monitor the temperature field and compaction uniformity of asphalt paving in harsh environments such as rainy days is not only of great theoretical value, but also of urgent practical significance for improving the construction quality and long-term durability of road engineering. Summary of the Invention

[0006] The purpose of this invention is to provide a method and system for monitoring the uniformity of asphalt paving in rainy weather based on multi-source sensor data. It aims to solve the key technical problem that infrared thermal imagers cannot accurately obtain the temperature distribution field during the asphalt paving process due to interference effects such as scattering and absorption of infrared radiation by rainwater and surface reflection in harsh rainy environments, which leads to the failure of real-time monitoring systems and affects construction quality and safety.

[0007] To achieve the above-mentioned objectives, the first objective of this invention is a method for monitoring the uniformity of asphalt paving in rainy weather based on multi-source sensor data, comprising: S1. Obtain the original infrared thermal image, original radar point cloud and environmental weather data stream of the asphalt paving process in rainy weather. Construct the original infrared thermal image sequence using the original infrared thermal image and the original radar point cloud sequence using the original radar point cloud. S2. Perform multimodal data synchronization and spatiotemporal tensor construction on the original infrared thermal image sequence, the original radar point cloud sequence, and the environmental weather data stream to obtain a multimodal spatiotemporal tensor sequence; S3. Depression detection is performed on the multimodal spatiotemporal tensor sequence to obtain the depression attention map; S4. Based on the depression attention map, perform physical constraint-based spatiotemporal network heatmap reconstruction on the multimodal spatiotemporal tensor sequence to obtain the reconstructed heatmap sequence. S5. Based on real-time weather data, road surface materials and structural parameters, the compaction window period is extrapolated from the reconstructed heat map sequence to obtain a future temperature prediction map. S6. Based on the current reconstructed temperature field and future temperature prediction map in the reconstructed heat map sequence, a compaction guidance map is generated to obtain the compaction guidance map.

[0008] Preferably, S3 includes: S301. Extract height map sequences from multimodal spatiotemporal tensor sequences; S302. Perform time smoothing on the height map sequence to obtain a stable height map; S303. Perform depression segmentation and geometric feature extraction based on marker watershed transformation on the stable height map to obtain a depression information set; S304. Perform water accumulation risk quantification and normalization attention map generation on the set of depression information to obtain the depression attention map.

[0009] Preferably, S303 includes: S3011. Perform terrain inversion on the stable elevation map to obtain the inverted elevation map; S3012. Generate foreground / background markers on the inverted terrain height map to obtain foreground markers, background markers, and unknown area markers; S3013. Using the foreground and background markers as seed points, perform the labeling watershed algorithm on the inverted topographic height map to obtain a labeled segmentation map; S3014. Traverse each unique label in the labeled segmentation map, extract depression information for each label, and obtain the depression information set.

[0010] Preferably, S3014 includes: S30141. Extract all pixels corresponding to the label to obtain the binary mask; S30142. Calculate the total number of pixels in the binary mask and use the total number as the area of ​​the depression; S30143. Calculate the average pixel value of the binary mask in the inverted terrain height map, and use the average pixel value as the average depth of the depression.

[0011] Preferably, S4 includes: S401. Input the multimodal spatiotemporal tensor sequence into the 3D convolutional encoder to obtain the encoder feature map; S402. Upsample or downsample the depression attention map to obtain a sampled depression attention map, wherein the sampled depression attention map has the same spatial resolution as the encoder feature map; S403. Generate a sampling offset field based on the encoder feature map and the post-sampling depression attention map; S404. Perform deformable convolution operation on the encoder feature map based on the sampling offset field to obtain the spatially aligned encoder feature map. S405. Based on the sampled depression attention map, gating fusion is performed on the encoder feature map and the spatially aligned encoder feature map to obtain the modulated encoder feature map. S406. Input the modulated encoder feature map into the 3D deconvolution decoder to obtain the reconstructed heat map sequence.

[0012] Preferably, in S403, the sampling offset field is generated using the following formula: ; in, For convolutional layers used to regress offsets, For encoder feature maps, This is the attention map of the depression after sampling. This is the feature channel splicing function. This represents the sampling offset field. Preferably, in S404, a deformable convolution operation is performed on the encoder feature map using the following formula: ; in, For encoder feature maps, This is a location in the spatially aligned encoder feature map. This represents the total number of sampling points in the convolution kernel. This represents the weight at the k-th position of the convolution kernel. The fixed offset of the k-th sampling point of the standard convolution kernel relative to the center. This represents the offset taken from the corresponding position in the sampling offset field. This indicates that the encoder feature map is spatially aligned at the location The value at that location.

[0013] Preferably, S5 includes: S501. Extract the heat map of the last time step from the reconstructed heat map sequence as the initial temperature field; S502. Input the current real-time weather data, road material and structural parameters, and the initial temperature field into the asphalt cooling analytical model to obtain a future temperature prediction map.

[0014] A second objective of this invention is to provide a monitoring system for the uniformity of asphalt paving in rainy weather based on multi-source sensor data, comprising: The data preparation module acquires the original infrared thermal image, original radar point cloud and environmental weather data stream of the asphalt paving process in rainy weather, constructs the original infrared thermal image sequence using the original infrared thermal image, and constructs the original radar point cloud sequence using the original radar point cloud. The multimodal spatiotemporal tensor construction module performs multimodal data synchronization and spatiotemporal tensor construction on the original infrared thermal image sequence, the original radar point cloud sequence, and the environmental weather data stream to obtain a multimodal spatiotemporal tensor sequence; The depression attention perception module detects depressions in multimodal spatiotemporal tensor sequences and obtains a depression attention map. The physical constraint heatmap reconstruction module, based on the depression attention map, performs physical constraint-based spatiotemporal network heatmap reconstruction on multimodal spatiotemporal tensor sequences to obtain the reconstructed heatmap sequence; The compaction window temperature prediction module extrapolates the compaction window period based on real-time weather data, road material and structural parameters, and obtains a future temperature prediction map. The compaction guidance map generation module generates a compaction guidance map based on the current reconstructed temperature field and the future temperature prediction map in the reconstructed heat map sequence.

[0015] A third objective of this invention is to provide a computer program product, including a computer program that is executed by a processor to perform the aforementioned method for monitoring the uniformity of asphalt paving in rainy weather based on multi-source sensor data.

[0016] A fourth objective of this invention is to provide a computer-readable storage medium including instructions that, when executed on a computer, cause the computer to perform the aforementioned method for monitoring the uniformity of asphalt paving in rainy weather based on multi-source sensor data.

[0017] Compared with the prior art, the present invention has the following technical effects: This invention effectively overcomes the fundamental flaw of existing infrared thermometry technology, which completely fails in rainy conditions due to the large-area absorption and scattering of infrared radiation by water and the interference of specular reflection. By fusing the original infrared thermal image and the original radar point cloud, and combining meteorological conditions for intelligent reconstruction under physical constraints, reliable recovery and continuous dynamic monitoring of the real temperature field during asphalt paving in rainy weather are achieved.

[0018] This invention transforms the monitoring model from a reactive, post-event quality recording approach to a proactive, pre-event decision-making guidance approach. By accurately predicting future temperatures, it can proactively calculate the remaining effective compaction time window for each paving point, providing quantitative and visual decision-making basis for roller path planning and compaction timing selection. This significantly avoids quality risks such as insufficient compaction caused by missing the optimal compaction temperature range.

[0019] The real-time compaction guidance map generated by this invention is intuitive and easy to understand with its heat map, which greatly reduces the difficulty of operation and judgment for construction personnel and reduces their over-reliance on personal experience. While ensuring the consistency and scientific nature of construction decisions, it comprehensively improves the uniformity of asphalt paving operations in rainy weather and the final quality of the road surface project. Attached Figure Description

[0020] Figure 1 A complete flowchart is provided for a preferred embodiment of the present invention; Figure 2This is a flowchart of parts S2-S6 in a preferred embodiment of the present invention; Figure 3 This is a flowchart of S3 in a preferred embodiment of the present invention; Figure 4 This is a flowchart of S4 in a preferred embodiment of the present invention; Figure 5 This is a flowchart of S5 in a preferred embodiment of the present invention; Figure 6 This is a system block diagram provided for a preferred embodiment of the present invention. Detailed Implementation

[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. The described embodiments are only a part of the embodiments of the present invention, and not all of them. Generally, the embodiments of the present invention described and shown in the accompanying drawings are characteristic technologies and solutions. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0022] For the first embodiment, please refer to... Figure 1 and Figure 2 A method for monitoring the uniformity of asphalt paving in rainy weather based on multi-source sensor data mainly includes the following six steps: S1. Obtain the original infrared thermal image, original radar point cloud and environmental weather data stream of the asphalt paving process in rainy weather. Construct the original infrared thermal image sequence using the original infrared thermal image and the original radar point cloud sequence using the original radar point cloud. During asphalt paving in rainy weather, raw infrared thermal images and raw radar point clouds are first acquired in real time using installed infrared thermal imagers and lidar. Environmental weather data streams are also obtained from weather stations. This data is used to comprehensively monitor temperature distribution and surface characteristics during the paving operation. Subsequently, based on chronological and spatial information, an infrared thermal image sequence is constructed using the acquired raw infrared thermal images to analyze temperature trends over time. Simultaneously, a radar point cloud sequence is constructed based on the raw radar point clouds to further assess the smoothness of the paved surface and identify potential defects.

[0023] The number and location of infrared thermal imagers, lidar, and weather stations can be set according to actual needs.

[0024] S2. Perform multimodal data synchronization and spatiotemporal tensor construction on the original infrared thermal image sequence, the original radar point cloud sequence, and the environmental weather data stream to obtain a multimodal spatiotemporal tensor sequence; In real-world monitoring scenarios, various sensors such as infrared thermal imagers, lidar, and weather stations operate independently with different frequencies, coordinate systems, and data formats. The data they acquire directly is fragmented and heterogeneous, making it unsuitable for subsequent fusion analysis and model processing. Therefore, a preprocessing workflow is needed to transform these raw data streams, which are independent in both time and space dimensions, into a structured data form with aligned information, namely, a multimodal spatiotemporal tensor sequence.

[0025] S3. Depression detection is performed on the multimodal spatiotemporal tensor sequence to obtain the depression attention map; In rainy paving scenarios, puddles are the primary cause of rapid and uneven cooling of the asphalt surface temperature, severely distorting temperature data captured by infrared thermal imagers in these areas. Therefore, to enable the subsequent thermal image reconstruction network to specifically and intelligently repair this severely contaminated data, clear guidance is needed beforehand. Specifically, by processing high-precision geometric data, a puddle attention map is generated that highlights these puddles and their risk levels. This map serves as strong prior knowledge, guiding the neural network to focus computational resources and attention on the locations most in need of repair, thus providing crucial spatial guidance information for achieving high-precision thermal image reconstruction.

[0026] Specifically, depression detection is not simply about finding low-lying areas, but a systematic process. It utilizes elevation information from the multimodal spatiotemporal tensor sequence constructed in the previous step, employing a series of image processing and feature extraction algorithms to automatically identify, segment, and quantify the geometric features (such as area and depth) of localized depressions on the paved surface that may lead to water accumulation. The final output of this step, the depression attention map, is a two-dimensional probability map or weighted map with the same spatial dimensions as the heatmap. Each pixel value in this map typically ranges from 0 to 1 and does not represent a physical quantity, but rather characterizes the probability that the pixel location belongs to a water-filled depression or the severity of its interference with the temperature field. A pixel value close to 1.0 indicates that the location is the core of a high-risk water-filled depression, while a pixel value close to 0 represents a flat or raised safe area.

[0027] S4. Based on the depression attention map, perform physical constraint-based spatiotemporal network heatmap reconstruction on the multimodal spatiotemporal tensor sequence to obtain the reconstructed heatmap sequence. Simple filtering or smoothing methods cannot handle the nonlinear temperature anomalies caused by waterlogged depressions. Therefore, a more intelligent model is needed that not only understands the spatiotemporal evolution of the data but also integrates prior knowledge from different modalities (i.e., the geometric location of the depressions) and adheres to basic physical principles (i.e., the laws of heat conduction), thereby achieving deep repair and reconstruction of damaged data. This step aims to construct a deep learning network guided by depression attention maps and constrained by physical laws to deduce a clean and physically realistic internal temperature field of asphalt from severely contaminated multimodal data. The spatiotemporal network refers to a deep neural network capable of processing data with both temporal and spatial dimensions. In this application, it is specifically embodied in a 3D convolution-based encoder-decoder structure (such as 3DU-Net), which can capture the dynamic changes of the temperature field in continuous time frames (time dimension) and a two-dimensional plane (spatial dimension). Physical constraints refer to the introduction of an additional loss term during the training of a neural network, in addition to using a small amount of real data for supervision. This loss term penalizes predictions that do not conform to preset physical laws (in this case, the partial differential equation of heat conduction), thereby forcing the network to generate physically reasonable outputs. Heatmap reconstruction is the direct output of this step. It is not a process of predicting the future, but rather a process of repairing, denoising, and correcting the input, contaminated heatmap sequence to obtain a clean heatmap sequence that is within the same time period as the input sequence but is numerically closer to the reality.

[0028] S5. Based on real-time weather data, road surface materials and structural parameters, the compaction window period is extrapolated from the reconstructed heat map sequence to obtain a future temperature prediction map. Simply knowing the current temperature distribution is insufficient for scientifically and optimally planning the timing and path of compaction operations, as the essence of construction decision-making is predicting future state evolution. This step introduces an analytical model based on physics principles, using the reconstructed high-quality thermal map as a precise initial condition. Under the combined influence of the real environmental boundary conditions defined by real-time weather data and the intrinsic material properties defined by pavement parameters, extrapolation calculations are performed over time. This process can accurately predict the temperature distribution at a specific future time point (e.g., 15 minutes later). The compaction window extrapolation is a deterministic calculation process based on first principles of physics. It does not employ statistical learning or black-box models for probabilistic prediction, but rather applies classical heat conduction theory, using analytical or semi-analytical mathematical models to solve the problem of future temperature evolution under specific boundary conditions for a given initial temperature field.

[0029] S6. Based on the current reconstructed temperature field and future temperature prediction map in the reconstructed heat map sequence, a compaction guidance map is generated to obtain the compaction guidance map.

[0030] This step involves in-depth processing and semantic transformation of temperature evolution data to directly calculate the remaining effective compaction time at each location and visualize it in a color-coded format, thereby building a bridge between advanced data analysis and actual front-line operations.

[0031] To better understand the technical concept of this invention, the following non-limiting description is provided: In one embodiment, S2 includes: performing time synchronization, spatial registration, gridding, feature extraction, and tensor construction on the acquired original infrared thermal image sequence, original radar point cloud sequence, and environmental weather data stream to finally obtain a multimodal spatiotemporal tensor sequence.

[0032] Specifically, time synchronization is the first step. Since different sensors acquire data at different frequencies—for example, infrared thermal imagers and lidar may acquire data at high frequencies, while weather stations may acquire data at low frequencies—it is essential to use a high-precision common clock source as a reference to timestamp all input data streams. Typically, a high-frequency Global Positioning System (GPS) clock is used as the reference to assign a precise, uniformly formatted timestamp to each frame of infrared thermal image, each frame of radar point cloud, and each weather data point. For data streams acquired at lower frequencies (such as weather data), time interpolation is used to calculate the corresponding values ​​at higher-frequency time points based on the values ​​of the preceding and following moments, thus ensuring that data from all modalities is available at any given time.

[0033] Next, time synchronization is performed. Each sensor is installed in a different location and has its own independent coordinate system. The purpose of spatial registration is to unify all sensor data into the same geographic coordinate system. This process utilizes sensor carrier trajectory data provided by high-precision GPS and inertial measurement units (IMUs) to project and transform the pixel coordinates of the infrared thermal image and the three-dimensional spatial coordinates of the radar point cloud into a pre-defined world coordinate system or vehicle coordinate system, ensuring the consistency of different data in spatial location.

[0034] Next, gridding and feature extraction are performed. On a unified spatiotemporal reference, continuous or discrete sensor data needs to be mapped onto a regular two-dimensional grid, the resolution of which is typically consistent with that of the infrared thermogram. For radar point cloud data, a height map of the same size as the infrared thermogram is generated by aggregating all three-dimensional points falling within each grid cell (e.g., taking the elevation mean or median). For global information such as weather data, the values ​​are broadcast to every grid cell at that moment.

[0035] Finally, tensor construction is performed. After the above processing, at each time step, each grid cell corresponds to a feature vector containing multiple information, such as [temperature value, altitude value, rainfall rate value, wind speed value]. Stacking the feature maps of all time steps within a selected time window (e.g., the past few seconds) along the time dimension forms the final multimodal spatiotemporal tensor sequence. This tensor has a clear structure and well-defined dimensions, and can be directly used as input for subsequent deep learning models.

[0036] To illustrate S2 more clearly, a specific embodiment will be used as an example below: During a paving operation, data acquisition was conducted first: an FLIRA65 infrared thermal imager continuously acquired surface thermal images of the paved asphalt sheet at a frequency of 15Hz, with a resolution of 640x512 pixels; simultaneously, a Velodyne VLP-16 lidar scanned the same area at a frequency of 15Hz to obtain high-precision 3D point cloud data; and a vehicle-mounted weather station acquired current ambient temperature, wind speed, and rainfall rate data at a frequency of 1Hz. All sensors were equipped with high-precision GPS / IMU modules, providing unified UTC timestamps and centimeter-level location information.

[0037] Data processing followed: First, spatiotemporal synchronization was performed, strictly aligning the infrared thermal image, radar point cloud, and weather data based on the GPS timestamp. Next, coordinate registration and meshing were performed, converting the radar point cloud data into a height map with the same resolution (640x512) as the infrared thermal image. Finally, tensor construction was performed, selecting data from the past 3 seconds (corresponding to 45 frames) to construct a multimodal spatiotemporal tensor sequence of size 45x4x640x512. In this tensor, the first dimension 45 represents the length of the time series, the second dimension 4 represents the number of feature channels, and these four channels correspond to the temperature (°C), relative surface height (cm), rainfall rate (mm / h), and wind speed (m / s) at each spatiotemporal point, respectively. The latter two dimensions 640 and 512 represent the spatial resolution.

[0038] like Figure 3 As shown, S3 includes: S301. Extract height map sequences from multimodal spatiotemporal tensor sequences; S302. Perform time smoothing on the height map sequence to obtain a stable height map; S303. Perform depression segmentation and geometric feature extraction based on marker watershed transformation on the stable height map to obtain a depression information set; S304. Perform water accumulation risk quantification and normalization attention map generation on the set of depression information to obtain the depression attention map.

[0039] S303 includes: S3011. Perform terrain inversion on the stable elevation map to obtain the inverted elevation map; S3012. Generate foreground / background markers on the inverted terrain height map to obtain foreground markers, background markers, and unknown area markers; S3013. Using the foreground and background markers as seed points, perform the labeling watershed algorithm on the inverted topographic height map to obtain a labeled segmentation map; S3014. Traverse each unique label in the labeled segmentation map, extract depression information for each label, and obtain the depression information set.

[0040] S3014 includes: S30141. Extract all pixels corresponding to the label to obtain the binary mask; S30142. Calculate the total number of pixels in the binary mask and use the total number as the area of ​​the depression; S30143. Calculate the average pixel value of the binary mask in the inverted terrain height map, and use the average pixel value as the average depth of the depression.

[0041] Specifically, firstly, a heightmap sequence is extracted from the multimodal spatiotemporal tensor sequence, and then the heightmap sequence is temporally smoothed to obtain a stable heightmap. This process separates the feature channels representing geometric information from the input multimodal spatiotemporal tensor sequence, forming a time-series heightmap. Since the paver may vibrate during its movement, causing instantaneous noise in a single frame of the heightmap, this heightmap sequence needs to be smoothed over time. In a specific embodiment, the Exponential Weighted Moving Average (EWMA) algorithm is used. By iteratively weighting and averaging the heightmaps of each frame in the sequence, random noise is filtered out, thereby generating a stable heightmap that reflects the true and stable geometry of the paved surface.

[0042] Secondly, depression segmentation and geometric feature extraction based on labeled watershed transformation are performed on the stable elevation map to obtain a depression information set. Specifically, this includes: performing terrain inversion on the stable elevation map to obtain an inverted elevation map; generating foreground / background markers on the inverted elevation map to obtain foreground markers, background markers, and unknown area markers; using the foreground and background markers as seed points, performing a labeled watershed algorithm on the inverted elevation map to obtain a labeled segmentation map; and traversing each unique label in the labeled segmentation map to extract depression information for each label to obtain the depression information set.

[0043] Specifically, the first step is to perform topographic inversion on the stable elevation map to obtain an inverted elevation map. Since standard watershed algorithms are used to find catchment basins, corresponding to local minima in the image, to identify depressions representing low-lying areas, the stable elevation map needs to be numerically inverted first, so that the original deepest depression becomes a local peak in the inverted elevation map. The calculation formula for this process is: ; in, This indicates a reversed terrain elevation map. This indicates an operation that retrieves the maximum value. This represents a stable height map.

[0044] The second step involves generating foreground / background markers on the inverted topographic height map to obtain foreground markers, background markers, and unknown area markers. To avoid the severe oversegmentation problem caused by traditional watershed algorithms, a marker-based strategy is adopted. By setting two thresholds, high and low, foreground markers that are definitely depression cores and background markers that are definitely not depressions are determined on the inverted topographic height map, respectively.

[0045] The third step involves using the foreground and background markers as seed points to perform a labeled watershed algorithm on the inverted topographic height map to obtain a labeled segmentation map. This algorithm expands from the marker points and eventually segments the entire image into several regions with unique labels, each region corresponding to an independent depression.

[0046] The fourth step involves traversing each unique label in the labeled segmentation map and extracting depression information for each label to obtain the depression information set. For each segmented depression region, all pixels corresponding to that label are extracted to obtain a binary mask; then, the total number of pixels within the binary mask is calculated as the depression area; and the average pixel value of the binary mask in the inverted terrain height map is calculated as the average depth of the depression. All these geometric features of the depressions are organized into a structured depression information set.

[0047] Finally, the water accumulation risk is quantified and normalized to generate a normalized attention map from the set of depression information to obtain the depression attention map. This process transforms the extracted discrete geometric features into a continuous attention map. First, for each depression in the set of depression information, its water accumulation risk score is calculated based on its area and depth using a weighted formula. A typical calculation formula is as follows:

[0048] in, It is the risk score of the k-th depression. and It is its area and depth. and These are weighting coefficients. This represents the maximum area normalization parameter. It is a preset scalar constant representing the maximum depression area that may occur under typical operating conditions. This parameter is used to normalize the area of ​​any depression to the interval between 0 and 1. This represents the maximum depth normalization parameter. It is a preset scalar constant representing the maximum depression depth that may occur under typical operating conditions. This parameter is used to normalize the average depth of any depression to the interval between 0 and 1. Then, the calculated risk score is filled into the pixel region of the corresponding depression to form an original risk map. Finally, to make the values ​​more suitable as input to the neural network, a non-linear sigmoid function is usually applied to normalize the entire original risk map, smoothly mapping its values ​​to the interval, thereby generating the final depression attention map.

[0049] In one specific embodiment, firstly, a 45-frame height map sequence is extracted from the spatiotemporal tensor sequence. An exponentially weighted moving average (EWMA) is applied to this sequence for temporal smoothing, with a smoothing factor α set to 0.2 to filter out instantaneous noise such as vehicle vibration, generating a stable height map representing stable geometry. Next, terrain inversion is performed on this stable height map, with a foreground threshold set to 80% of the maximum value of the inverted terrain height map and a background threshold set to 20%, to generate foreground and background markers. Subsequently, using the foreground and background markers as seed points, a watershed labeling algorithm is executed on the inverted terrain height map to obtain a depression segmentation map with unique labels. Then, each label in the segmentation map is traversed, and its corresponding pixel set is extracted to form a binary mask. The total number of pixels within the mask is calculated to obtain the depression area. The average depth of the depression is obtained by calculating the average pixel value of the mask in the inverted terrain height map. This process ultimately forms a set of depression information, such as [{id:1,area:1250px,depth:1.8cm},{id:2,area:870px,depth:0.9cm},...]. Finally, for each depression in the set, a water accumulation risk score is calculated based on its area and depth. This score is then filled back into the corresponding depression's pixel region, and normalized using the Sigmoid function to generate the final depression attention map. In this map, depressions with higher risk have pixel values ​​closer to 1.0, while flat areas are closer to 0.

[0050] In S4, based on the depression attention map, a physically constrained spatiotemporal network heatmap reconstruction is performed on the multimodal spatiotemporal tensor sequence to obtain the reconstructed heatmap sequence. It can be understood that, in one embodiment, the physically constrained spatiotemporal network heatmap reconstruction employs a spatiotemporal convolutional network based on the 3DU-Net architecture to perform the heatmap reconstruction task. This network consists of a symmetric encoder-decoder structure, containing four downsampling modules and four upsampling modules. The encoder part progressively extracts multi-scale spatiotemporal features of the input spatiotemporal tensor through a series of 3D convolutional layers (with kernel size of 3x3x3) and 3D max-pooling layers, while reducing the spatial resolution and increasing the number of feature channels. The decoder part progressively restores the low-resolution depth feature map to the original spatial resolution through a series of 3D deconvolutional layers (or transposed convolutional layers) and 3D convolutional layers.

[0051] The application of the depression attention map in this embodiment is implemented through a direct feature modulation method. Skip connections exist between the encoder and decoder in the 3DU-Net. These connections directly pass the feature maps extracted by the encoder at different scales to the corresponding layers of the decoder to preserve spatial detail. In this embodiment, before passing the encoder's feature map to the decoder via skip connections, the depression attention map is downsampled to the same spatial resolution as the current encoder feature map using bilinear interpolation. Subsequently, the downsampled attention map is broadcast along the feature channel dimension, making it have the exact same size as the encoder feature map, and then element-wise multiplied with the encoder feature map. This operation is equivalent to a spatial gating mechanism, which selectively suppresses or allows information in the encoder feature map based on the value of the depression attention map. In depression regions (attention map values ​​close to 1), the feature weights are enhanced or maintained, guiding the network to focus on them; while in flat regions (attention map values ​​close to 0), the feature weights are weakened, indicating that the original temperature data in this region is relatively reliable and does not require significant correction.

[0052] During the network training phase, this embodiment employs a hybrid loss function for model optimization. This loss function consists of a data loss term. and physical loss item Weighted composition. Data loss item. The mean square error is calculated. Several (e.g., 10) thermocouple temperature probes are pre-embedded in the asphalt paving area. These probes provide sparse but absolutely accurate true ground temperature values. The data loss is the mean square error between the predicted temperature values ​​and the actual measured values ​​at the corresponding spatiotemporal coordinate points of the reconstructed heatmap sequence output by the network and these probes.

[0053] Physical loss item It does not rely on the true labels. It approximates the time partial derivatives of the temperature field by applying numerical differentiation (e.g., using finite-difference convolution kernels) to the reconstructed heatmap sequence. Second-order partial derivatives in space (Laplace operator) Then, these calculated gradient terms are substituted into the simplified heat conduction equation. This yields the physical residual at each spatiotemporal point. The physical loss is the mean square value of the physical residuals at all spatiotemporal points. Among these, the thermal diffusivity... Depending on the type of asphalt mixture used, a known physical constant is set (e.g., 5.5 x 10⁻⁶). -7 m 2 / s).

[0054] Total loss function The weighting coefficient In this embodiment, it is set to 0.05. By using the Adam optimizer to minimize this total loss function with a learning rate of 0.001, the network is trained to generate a temperature field that can both fit sparse real observation data and obey the laws of heat conduction globally.

[0055] During the actual execution (inference) phase, the trained network receives real-time multimodal spatiotemporal tensor sequences and depression attention maps. Through one forward propagation, it directly outputs a reconstructed heatmap sequence. The temperature distribution in this sequence is smooth and continuous, and areas in the original heatmap that exhibited severe low-temperature anomalies due to rain interference have been restored to more reasonable temperature values ​​that are consistent with their surrounding environment and physical laws.

[0056] In particular, considering the fusion of geometric prior information in spatiotemporal networks to correct distorted sensor data, the core challenge lies in achieving efficient alignment and deep interaction between information of different modalities and different levels of abstraction. In the aforementioned modulation schemes, the downsampled depression attention map is typically combined with the encoder's deep feature map through simple element-wise multiplication, which has inherent technical limitations. This limitation stems from the inconsistency in spatial semantics: the feature map output by the encoder after multiple convolutions and pooling corresponds to a broad receptive field in the original input space for each feature vector, carrying highly abstract, regional semantic information; while the depression attention map depicts precise, pixel-level local geometry. Rigidly multiplying a precise local geometric indicator directly with a vague regional semantic feature is essentially a coarse spatial alignment that fails to fully exploit the deep correlation between the two. Furthermore, this simple gating mechanism limits the information interaction mode. It can only adjust the pass rate of the original features, but cannot guide the network to adaptively learn more complex feature fusion strategies based on differences in geometric shape. For example, for irregularly shaped depressions, more refined operations should be performed, such as focusing on the temperature gradient features of their boundary regions.

[0057] To overcome this bottleneck, a novel modulation mechanism is introduced, its technical principle rooted in dynamic alignment and adaptive fusion. This mechanism no longer passively accepts the fixed grid structure of feature maps, but instead, by introducing deformable convolutions, empowers the network to actively learn sampling positions, thereby achieving precise spatial alignment between geometric shapes and semantic features. Simultaneously, by designing a gated fusion network, the model is given dynamic decision-making power, enabling it to intelligently determine the extent to which geometric information is adopted based on the context, achieving adaptive deep fusion of features.

[0058] In a preferred embodiment, such as Figure 4 As shown, S4 includes: S401. Input the multimodal spatiotemporal tensor sequence into the 3D convolutional encoder to obtain the encoder feature map; S402. Upsample or downsample the depression attention map to obtain a sampled depression attention map, wherein the sampled depression attention map has the same spatial resolution as the encoder feature map; S403. Generate a sampling offset field based on the encoder feature map and the post-sampling depression attention map; S404. Perform deformable convolution operation on the encoder feature map based on the sampling offset field to obtain the spatially aligned encoder feature map. S405. Based on the sampled depression attention map, gating fusion is performed on the encoder feature map and the spatially aligned encoder feature map to obtain the modulated encoder feature map. S406. Input the modulated encoder feature map into the 3D deconvolution decoder to obtain the reconstructed heat map sequence.

[0059] In this preferred embodiment, firstly, the multimodal spatiotemporal tensor sequence is input into a 3D convolutional encoder to obtain encoder feature maps. This encoder, through multiple layers of 3D convolution and pooling operations, extracts abstract spatiotemporal features from the input spatiotemporal tensor, ranging from low-level texture to high-level semantics, layer by layer, forming a series of encoder feature maps with different resolutions.

[0060] Then, the sampling offset field is dynamically generated. It should be understood that depressions formed on asphalt pavers due to water accumulation often exhibit irregular, non-mesh-aligned shapes. Standard convolution operations, using fixed rectangular sampling areas, cannot accurately capture the features of these irregular boundaries. Therefore, the network must learn to generate a customized sampling scheme for specific depression shapes. Specifically, the feature map output by the encoder at layer l is... Attention map of depressions obtained after spatial dimension adjustment The data is concatenated along the channel dimension to form a fused feature tensor containing both semantic and geometric information. This tensor is then fed into a dedicated lightweight convolutional network. The output of this network is the sampling offset field. In real-world asphalt paving monitoring scenarios, the network can identify a crescent-shaped water accumulation area and autonomously learn a sampling point offset pattern that is also crescent-shaped. This allows subsequent feature extraction operations to focus entirely on the irregular depression itself and its most relevant surrounding areas, laying the foundation for accurate feature space alignment.

[0061] In one embodiment, generating a sampling offset field based on the encoder feature map and the sampled depression attention map includes: generating the sampling offset field using the following formula, wherein the formula is: ; in, For convolutional layers used to regress offsets, For encoder feature maps, This is the attention map of the depression after sampling. This is the feature channel splicing function. This represents the sampling offset field.

[0062] Next, deformable convolution is performed based on the offset field to complete feature alignment sampling. That is, after obtaining a customized sampling scheme (offset field), an operator capable of executing this scheme is needed to complete the actual feature extraction. Deformable convolution breaks the rigid structure of traditional convolution kernels. Specifically, the offset field generated in the previous step... This is applied to a standard deformable convolutional layer, which uses the encoder's original feature map. As input, when computing each location in the output feature map, the sampling points of the convolutional kernel are no longer fixed grid points, but rather, based on the grid points, plus... Provided in the corresponding position The offset is used to sample at a dynamically adjusted, irregular location. In practical terms, for elongated strips of water formed by vehicles during paving, this operation allows feature extraction to precisely follow this elongated trajectory, rather than sampling features from a large number of irrelevant, flat areas. This generates a completely new feature map. The information in this feature map has been spatially reorganized, and its content is highly correlated with the actual geometric shape of the depression. It is a precise projection of geometric prior knowledge into the semantic feature space.

[0063] In one embodiment, performing a deformable convolution operation on the encoder feature map based on a sampling offset field to obtain a spatially aligned encoder feature map includes: performing a deformable convolution operation on the encoder feature map based on the sampling offset field using the following formula: ; in, For encoder feature maps, This is a location in the spatially aligned encoder feature map. This represents the total number of sampling points in the convolution kernel. This represents the weight at the k-th position of the convolution kernel. The fixed offset of the k-th sampling point of the standard convolution kernel relative to the center. This represents the offset taken from the corresponding position in the sampling offset field. This indicates that the encoder feature map is spatially aligned at the location The value at that location.

[0064] Next, a gating mechanism is used to achieve adaptive feature fusion. It's understandable that simply adding or concatenating the original semantic features with the aligned geometric features is a static fusion strategy that cannot adapt to complex scene changes. The network needs the ability to determine the relative importance of geometric and semantic information in a specific feature channel at the current location.

[0065] Specifically, construct a parallel gating network. Similarly, the spliced and The input is a sigmoid function, but its output is activated by the sigmoid function to generate a gated weight with a value range between [0,1]. This gating weight It was then used to analyze the original features. and aligned geometric features Weighted fusion is then performed. In a specific paving scenario, for a newly formed shallow puddle, its impact on the internal temperature is not yet significant. In this case, the gating network may learn to generate a weight close to 0, allowing the final output to retain more of the original semantic features reflecting the global temperature trend. However, for a long-standing deep puddle that has clearly caused a sudden drop in temperature, the gating weight may be close to 1, making the output features completely dominated by the aligned geometric features. The ultimate goal and effect of this step is to produce a refined, dynamically modulated feature map. It is neither purely semantic information nor purely geometric information, but rather the optimal combination of the two in the current spatiotemporal context, providing the highest quality input for the decoder's subsequent high-precision heat map reconstruction task.

[0066] The above process can be expressed by the following formula:

[0067]

[0068] in, It is the generated gated weight tensor; Represents the Sigmoid activation function; It is a convolutional layer used to generate gated weights. It is the final output modulated feature map; This represents element-wise multiplication operations.

[0069] The improved mechanism achieves a high degree of consistency between its technical effect and objective. It aims to seamlessly integrate geometric prior knowledge into spatiotemporal deep learning networks in a more refined and intelligent manner, thereby significantly improving the reconstruction accuracy of sensor data under harsh conditions. By introducing deformable convolutions, high-precision spatial alignment of irregular depression morphologies is achieved, ensuring that the model can extract information from the most relevant feature regions—the foundation for high-precision reconstruction. Furthermore, through a gated fusion network, the model is endowed with the ability to dynamically adjust the importance of different information sources based on real-time data, achieving a leap from static fusion to adaptive intelligent fusion, greatly enhancing the model's expressive power and robustness to complex dynamic scenarios. This deep alignment and fusion not only directly improves the accuracy of the final heatmap reconstruction, making subsequent compaction window predictions more reliable, but also enhances the model's interpretability due to its inherent structured design. By analyzing the offset field and gate weights, the key basis for the model's decision-making can be discerned, thus achieving a technical objective that is both efficient and reliable.

[0070] like Figure 5 As shown, S5 includes: S501. Extract the heat map of the last time step from the reconstructed heat map sequence as the initial temperature field; S502. Input the current real-time weather data, road material and structural parameters, and the initial temperature field into the asphalt cooling analytical model to obtain a future temperature prediction map.

[0071] Specifically, firstly, the heatmap from the last time step of the reconstructed heatmap sequence is extracted as the initial temperature field. Since the reconstructed heatmap sequence is arranged chronologically, the image from the last time step represents the most accurate and realistic temperature distribution of the asphalt surface at the current moment. The system extracts this image as a two-dimensional initial temperature field. This provides the starting state for subsequent cooling prediction calculations.

[0072] After obtaining the accurate initial temperature field, the current real-time weather data, road material and structural parameters, and the initial temperature field are input into the asphalt cooling analytical model to obtain a future temperature prediction map. This is a point-by-point, parallel calculation process. For each pixel in the initial temperature field... Each calculation performs a cooling prediction independently. The core of this calculation is the application of a mature asphalt cooling analytical model, such as the Barber model. This model first calculates the key parameter describing the intensity of surface heat exchange, namely the overall surface heat transfer coefficient, based on the input real-time weather data and pavement material parameters. This coefficient is typically determined by the convective heat transfer coefficient. and radiation heat transfer coefficient It consists of two parts. The convective heat transfer coefficient is mainly affected by wind speed, while the radiative heat transfer coefficient is related to the road surface emissivity and surface temperature. Subsequently, the model sets the initial temperature of this pixel. Calculated surface heat transfer coefficient Ambient temperature The thermal parameters of the material (such as thermal conductivity, specific heat capacity, and density) are then substituted into the analytical or semi-analytical solution of the heat conduction equation. By solving this equation, the predicted temperature value of the pixel can be obtained after a preset time length (e.g., 15 minutes) from the current moment. Finally, the calculated future temperature values ​​for all pixels are recombined to form a complete future temperature prediction map with the same size as the initial temperature field.

[0073] In one specific embodiment, based on current real-time weather data and road material and structural parameters, the compaction window period is extrapolated from the reconstructed heat map sequence to obtain a future temperature prediction map. This includes: first, extracting the heat map of the last time step from the reconstructed heat map sequence as the most accurate initial temperature field. Then, a cooling model prediction is performed. The system will... As initial conditions, real-time weather data (specifically, ambient temperature 18°C, wind speed 4 m / s) and pavement material and structural parameters (specifically, a certain mixture, layer thickness 6 cm, thermal conductivity 1.5 W / (m·K), specific heat capacity 1700 J / (kg·K), etc.) are used as boundary conditions. Then, all the above data are input into a pre-defined asphalt cooling analytical model (e.g., the Barber model). This model calculates the temperature of each point in the paved area precisely 15 minutes later by solving the heat conduction equation, ultimately generating a future temperature prediction map.

[0074] S6 includes: determining the remaining effective compaction window period based on the current reconstructed temperature field and future temperature prediction map, obtaining a compaction window period map with remaining compaction minutes, and visualizing and publishing the compaction window period map with remaining compaction minutes.

[0075] Specifically, the first step is to calculate the remaining effective compaction window period. This process requires pre-setting a minimum effective compaction temperature threshold for the current asphalt mixture, which is the critical temperature to ensure compaction quality. For each pixel in the paving area... The system will determine the temperature value based on the temperature value in the current reconstructed temperature field. and temperature values ​​in future temperature prediction maps. The temperature at that point was calculated using interpolation. Descending to Time required In a simplified model assuming a linear temperature decrease over a certain time period, the formula for calculating this time period is: ; in, This is the time span (e.g., 15 minutes) corresponding to the future temperature forecast map. By performing this calculation on all pixels on the map, a complete compaction window map can be obtained, where each pixel value represents the remaining compaction minutes.

[0076] The second step is map visualization and publishing. This process transforms the continuous numerical time window map into a color-coded, easily perceptible, compacted guidance map. The system will, based on a preset time threshold, specify the remaining window period for each pixel. The data is mapped to specific colors. For example, a three-level color coding scheme can be set: if the remaining window is greater than 15 minutes, the pixel is displayed in green, indicating that the compaction operation is safe; if the remaining window is between 5 and 15 minutes, it is displayed in yellow, indicating that compaction needs to be arranged as soon as possible; if the remaining window is less than 5 minutes, it is displayed in red, indicating that final compaction must be carried out immediately, otherwise there is a serious quality risk. Finally, this color-coded compaction guidance map is pushed in real time wirelessly to all terminal devices such as flat panel displays in the cabs of all road rollers on site for operators to view and use.

[0077] In one specific embodiment, a compaction guidance map is generated based on the current reconstructed temperature field and future temperature prediction map in the reconstructed heat map sequence. This includes: First, calculating the window period: setting the minimum effective compaction temperature threshold for a certain material to 90°C. For each pixel in the paving area, the time required for the temperature at that point to drop from the current value to 90°C is calculated by interpolation based on its current reconstructed temperature and future temperature prediction map. This time is the remaining effective compaction window period for that point. Then, map visualization and publishing are performed: a color-coded compaction guidance map is generated based on the calculated remaining window period. The specific coding rules are as follows: green areas represent a remaining window period greater than 15 minutes, indicating safe compaction operations; yellow areas represent a remaining window period between 5 and 15 minutes, requiring immediate compaction; red areas represent a remaining window period less than 5 minutes, requiring immediate final compaction, otherwise there is a quality risk. The guidance map is pushed in real time to the flat panel displays in the cabs of all road rollers on site, guiding operators to optimize the compaction path and sequence, and prioritizing the treatment of yellow and red areas that are about to "expire", thereby ensuring high-quality and high-uniformity compaction operations even in rainy conditions.

[0078] This invention first synchronizes and constructs a spatiotemporal tensor from the original infrared thermal image sequence, the original radar point cloud sequence, and the environmental weather data stream using multimodal data. Second, it utilizes the geometric information in the spatiotemporal tensor to detect depressions, generating a depression attention map marking waterlogged areas. The core of this invention lies in guiding a spatiotemporal neural network with built-in physical constraints to process the multimodal spatiotemporal tensor, reconstructing a thermal image sequence unaffected by rainwater. Subsequently, the reconstructed current temperature field is used as a precise initial condition, combined with real-time weather and material parameters, to extrapolate future compaction windows through an analytical model. Finally, an intuitive and visual compaction guidance map is generated. This enables reliable recovery and forward-looking prediction of asphalt paving temperature under rainy conditions, significantly improving construction quality and the level of intelligent monitoring.

[0079] like Figure 6As shown, a rain-weather asphalt paving uniformity monitoring system based on multi-source sensor data is used to implement the method of the first embodiment described above, comprising: The data preparation module acquires the original infrared thermal image, original radar point cloud and environmental weather data stream of the asphalt paving process in rainy weather, constructs the original infrared thermal image sequence using the original infrared thermal image, and constructs the original radar point cloud sequence using the original radar point cloud. The multimodal spatiotemporal tensor construction module performs multimodal data synchronization and spatiotemporal tensor construction on the original infrared thermal image sequence, the original radar point cloud sequence, and the environmental weather data stream to obtain a multimodal spatiotemporal tensor sequence; The depression attention perception module detects depressions in multimodal spatiotemporal tensor sequences and obtains a depression attention map. The physical constraint heatmap reconstruction module, based on the depression attention map, performs physical constraint-based spatiotemporal network heatmap reconstruction on multimodal spatiotemporal tensor sequences to obtain the reconstructed heatmap sequence; The compaction window temperature prediction module extrapolates the compaction window period based on real-time weather data, road material and structural parameters, and obtains a future temperature prediction map. The compaction guidance map generation module generates a compaction guidance map based on the current reconstructed temperature field and the future temperature prediction map in the reconstructed heat map sequence.

[0080] The third embodiment is a computer program product, including a computer program that is executed by a processor as described above for monitoring the uniformity of asphalt paving in rainy weather based on multi-source sensor data.

[0081] Fourth embodiment: A computer-readable storage medium includes instructions that, when executed on a computer, cause the computer to perform the above-described method for monitoring the uniformity of asphalt paving in rainy weather based on multi-source sensor data.

[0082] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented, in whole or in part, as a computer program product, the computer program product includes one or more computer instructions. When the computer program instructions are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)).

[0083] The above description is only a preferred embodiment of the present invention. It should be noted that any improvements, modifications, substitutions or variations made by those skilled in the art without departing from the principle of the present invention should be considered as being included within the protection scope of the present invention.

Claims

1. A method for monitoring the uniformity of asphalt paving in rainy weather based on multi-source sensor data, characterized in that, include: S1. Obtain the original infrared thermal image, original radar point cloud and environmental weather data stream of the asphalt paving process in rainy weather. Construct the original infrared thermal image sequence using the original infrared thermal image and the original radar point cloud sequence using the original radar point cloud. S2. Perform multimodal data synchronization and spatiotemporal tensor construction on the original infrared thermal image sequence, the original radar point cloud sequence, and the environmental weather data stream to obtain a multimodal spatiotemporal tensor sequence; S3. Depression detection is performed on the multimodal spatiotemporal tensor sequence to obtain the depression attention map; S4. Based on the depression attention map, perform physical constraint-based spatiotemporal network heatmap reconstruction on the multimodal spatiotemporal tensor sequence to obtain the reconstructed heatmap sequence. S5. Based on real-time weather data, road surface materials and structural parameters, the compaction window period is extrapolated from the reconstructed heat map sequence to obtain a future temperature prediction map. S6. Based on the current reconstructed temperature field and future temperature prediction map in the reconstructed heat map sequence, a compaction guidance map is generated to obtain the compaction guidance map.

2. The method for monitoring the uniformity of asphalt paving in rainy weather based on multi-source sensor data according to claim 1, characterized in that, S3 include: S301. Extract height map sequences from multimodal spatiotemporal tensor sequences; S302. Perform time smoothing on the height map sequence to obtain a stable height map; S303. Perform depression segmentation and geometric feature extraction based on marker watershed transformation on the stable height map to obtain a depression information set; S304. Perform water accumulation risk quantification and normalization attention map generation on the set of depression information to obtain the depression attention map.

3. The method for monitoring the uniformity of asphalt paving in rainy weather based on multi-source sensor data according to claim 2, characterized in that, S303 includes: S3011. Perform terrain inversion on the stable elevation map to obtain the inverted elevation map; S3012. Generate foreground / background markers on the inverted terrain height map to obtain foreground markers, background markers, and unknown area markers; S3013. Using the foreground and background markers as seed points, perform the labeling watershed algorithm on the inverted topographic height map to obtain a labeled segmentation map; S3014. Traverse each unique label in the labeled segmentation map, extract depression information for each label, and obtain the depression information set.

4. The method for monitoring the uniformity of asphalt paving in rainy weather based on multi-source sensor data according to claim 3, characterized in that, S3014 includes: S30141. Extract all pixels corresponding to the label to obtain the binary mask; S30142. Calculate the total number of pixels in the binary mask and use the total number as the area of ​​the depression; S30143. Calculate the average pixel value of the binary mask in the inverted terrain height map, and use the average pixel value as the average depth of the depression.

5. The method for monitoring the uniformity of asphalt paving in rainy weather based on multi-source sensor data according to any one of claims 1-4, characterized in that, S4 include: S401. Input the multimodal spatiotemporal tensor sequence into the 3D convolutional encoder to obtain the encoder feature map; S402. Upsample or downsample the depression attention map to obtain a sampled depression attention map, wherein the sampled depression attention map has the same spatial resolution as the encoder feature map; S403. Generate a sampling offset field based on the encoder feature map and the post-sampling depression attention map; S404. Perform deformable convolution operation on the encoder feature map based on the sampling offset field to obtain the spatially aligned encoder feature map. S405. Based on the sampled depression attention map, gating fusion is performed on the encoder feature map and the spatially aligned encoder feature map to obtain the modulated encoder feature map. S406. Input the modulated encoder feature map into the 3D deconvolution decoder to obtain the reconstructed heat map sequence.

6. The method for monitoring the uniformity of asphalt paving in rainy weather based on multi-source sensor data according to claim 5, characterized in that, In S403, the sampling offset field is generated using the following formula: ； in, For convolutional layers used to regress offsets, For encoder feature maps, This is the attention map of the depression after sampling. This is the feature channel splicing function. This represents the sampling offset field.

7. The method for monitoring the uniformity of asphalt paving in rainy weather based on multi-source sensor data according to claim 5, characterized in that, In S404, deformable convolution operations are performed on the encoder feature maps using the following formula: ； in, For encoder feature maps, This is a location in the spatially aligned encoder feature map. This represents the total number of sampling points in the convolution kernel. This represents the weight at the k-th position of the convolution kernel. The fixed offset of the k-th sampling point of the standard convolution kernel relative to the center. This represents the offset taken from the corresponding position in the sampling offset field. This indicates that the encoder feature map is spatially aligned at the location The value at that location.

8. The method for monitoring the uniformity of asphalt paving in rainy weather based on multi-source sensor data according to claim 1, characterized in that, S5 include: S501. Extract the heat map of the last time step from the reconstructed heat map sequence as the initial temperature field; S502. Input the current real-time weather data, road material and structural parameters, and the initial temperature field into the asphalt cooling analytical model to obtain a future temperature prediction map.

9. A monitoring system for the uniformity of asphalt paving in rainy weather based on multi-source sensor data, characterized in that, include: The data preparation module acquires the original infrared thermal image, original radar point cloud and environmental weather data stream of the asphalt paving process in rainy weather, constructs the original infrared thermal image sequence using the original infrared thermal image, and constructs the original radar point cloud sequence using the original radar point cloud. The multimodal spatiotemporal tensor construction module performs multimodal data synchronization and spatiotemporal tensor construction on the original infrared thermal image sequence, the original radar point cloud sequence, and the environmental weather data stream to obtain a multimodal spatiotemporal tensor sequence; The depression attention perception module detects depressions in multimodal spatiotemporal tensor sequences and obtains a depression attention map. The physical constraint heatmap reconstruction module, based on the depression attention map, performs physical constraint-based spatiotemporal network heatmap reconstruction on multimodal spatiotemporal tensor sequences to obtain the reconstructed heatmap sequence; The compaction window temperature prediction module extrapolates the compaction window period based on real-time weather data, road material and structural parameters, and obtains a future temperature prediction map. The compaction guidance map generation module generates a compaction guidance map based on the current reconstructed temperature field and the future temperature prediction map in the reconstructed heat map sequence.

10. A computer-readable storage medium comprising instructions, when executed on a computer, causing the computer to perform the method for monitoring the uniformity of asphalt paving in rainy weather based on multi-source sensor data as described in any one of claims 1-8.

Images