High-cold anti-freezing road surface state evaluation method based on deep learning

Abstract

The application belongs to the technical field of deep learning, and particularly relates to a high-cold anti-coagulation ice road surface state evaluation method based on deep learning. The method acquires a road surface image sequence, a road surface temperature sequence, an environment temperature and humidity sequence and a precipitation state record in a continuous low-temperature event of the same road section, forms a multi-source time sequence sample, extracts spatial features of road surface texture, reflective area, frost and snow particles and rut residual water boundary through a visual depth network, extracts time sequence features of low-temperature duration, temperature recovery interval, humidity mutation and precipitation change through a time sequence depth network, fuses the spatial features and the time sequence features in a conditional attention manner to obtain ice-related features, and according to an ice phase transition relationship constraint state probability between dryness, humidity, supercooled water film, nascent thin ice, stable ice film and ice melting recovery, outputs a road surface state evaluation result. The application can reduce the confusion between thin ice, wet reflective, ice melting residual liquid and frost layer.

Description

Technical Field

[0001] This invention belongs to the field of deep learning technology, specifically relating to a deep learning-based method for assessing the condition of cold-resistant and icy road surfaces. Background Technology

[0002] In high-altitude and cold regions, the assessment of pavement condition against icing typically relies on pavement images, surface temperature, ambient temperature and humidity, and precipitation records. Monitoring data is generated according to the collection time, and the pavement condition is then determined through manual interpretation, threshold judgment, or deep learning classification models. Conventional image processing methods generally involve first converting the acquired images to grayscale, denoising, enhancing them, and cropping the pavement area. Then, features such as texture, brightness, reflectivity, edge contours, and color distribution are extracted. Alternatively, the images are directly input into convolutional neural networks or visual transformation networks for classification training. During the training phase, categories such as dry, wet, snow-covered, and icy are usually set based on manually labeled samples, allowing the model to learn the apparent differences between different image categories. In the application phase, single frames or short sequences of images are input into the model to obtain corresponding state labels or state probabilities. To avoid background interference, some methods also include a pavement area separation step, excluding the sky, vehicles, roadside facilities, and unpaved areas from the image before feeding the remaining areas into the classification network. The key technical focus of this type of solution is to identify road surface coverings and morphology from visible images. It has a certain ability to distinguish between significantly different conditions such as snow cover, obvious ice layers, and dry, exposed surfaces. However, its judgment is still based on the image appearance itself. When encountering thin water films, melted ice residue, or the initial stage of frost formation, the differences in brightness and texture in the image are weak, and conventional visual models need to rely on a large number of labeled samples to maintain classification boundaries.

[0003] Another conventional approach focuses on environmental condition assessment, typically using road surface temperature, air temperature, relative humidity, precipitation type, and duration as inputs. It establishes a rule base based on temperature and humidity ranges near freezing point and precipitation duration, or uses time series models to predict whether icing conditions will occur in the short term. To improve the error of a single threshold, some approaches incorporate continuous cooling rate, cumulative duration below zero, warming intervals, and precipitation intensity into the assessment process, combining factors such as persistent low temperatures, road surface water, and rising humidity into an icing risk level. In the maintenance of high-altitude and cold-weather roads, this type of approach usually updates the temperature and humidity sequences at a fixed sampling period and then outputs results such as safe, cautious, or prone to icing based on preset rules. If deep learning methods are introduced, a common practice is to input the temperature and humidity sequences into a recurrent network, a temporal convolutional network, or a gated network, allowing the network to learn the correlation between environmental variables and historical states. Such schemes can reflect the driving process of external environment on road surface icing, but their output mainly depends on the environmental variables themselves. They cannot directly identify whether thin ice, frost, water film, or melt ice residue has formed on the actual road surface, nor can they distinguish the state differences between anti-icing pavement and ordinary pavement under the same low temperature conditions. When anti-icing pavement experiences delayed icing or melt ice recovery at low temperatures, a simple environmental sequence may easily equate the presence of icing conditions with actual icing.

[0004] In deep learning fusion evaluation schemes, a relatively close approach is to concatenate image classification results with temporal features of temperature and humidity, and then output the final state through a fully connected network or recurrent network. Such schemes typically use visual features as apparent information and temperature and humidity features as auxiliary information, merging them at the feature layer or decision layer to reduce misjudgments caused by the influence of illumination, shadows, and reflections on a single image. Some schemes also employ multi-task output, simultaneously providing the current state and short-term risk level, or smoothing the prediction results of adjacent frames to prevent frequent jumps in state changes. Although the above fusion methods incorporate multi-source data, most still use statistical correlation as the training objective. Temperature and humidity features are merely appended to image features without providing conditional constraints on specific reflective boundaries, frost particles, and rut water contours in the image. For continuously changing states such as low temperature and humidity, initial thin ice, stable ice film, and ice melt recovery, simple feature concatenation is insufficient to express the temporal phase transition relationship of the road surface state, and it is also difficult to use the state lag caused by anti-icing effects to correct the classification results. Therefore, in continuous low-temperature events, contradictory outputs of states at adjacent times may still occur. Smoothing can only suppress probability fluctuations, but cannot limit the reasonable transition path from wet to thin ice, and then to melting ice recovery.

[0005] The main technical problem with existing technologies is that, in continuous low-temperature events, cold-resistant anti-icing pavements exhibit similar appearances but different ice phases, leading to confusion. Simple image recognition or ordinary multi-source stitching models struggle to reliably determine the true pavement state. Specifically, thin ice, wet reflections, meltwater residue, and frost layers may all appear as gray-white coverage, bright spot expansion, or local texture loss in images, making visual models prone to mistaking similar appearances for the same state. While temperature and humidity can describe icing conditions, they cannot characterize the delaying effect of anti-icing materials on the icing process. Judging solely by temperature thresholds would directly classify pavements still in the supercooled water film or meltwater recovery stage as icy. Ordinary fusion models lack transition constraints between dryness, wetness, supercooled water film, nascent thin ice, stable ice film, and meltwater recovery, easily resulting in outcomes that do not conform to the evolution of low-temperature events, such as jumping directly from dryness to stable ice film or vice versa. This problem results in a lack of continuity in the assessment results across the boundary state, phase change state, and recovery state, making it difficult to reflect the icing lag characteristics of anti-icing pavement compared to ordinary pavement. Furthermore, it makes it difficult to establish a unified state sequence basis for subsequent short-term icing risk levels and the degree of lag in anti-icing effects, thus making it difficult to form a verifiable temporal judgment basis for continuous sampling results of the same road segment. Summary of the Invention

[0006] The purpose of this invention is to provide a deep learning-based method for assessing the condition of cold-resistant and icy road surfaces, which can solve the problems mentioned in the background art.

[0007] To achieve the above objectives, the technical solution adopted by the present invention is as follows: A deep learning-based method for assessing the condition of cold-resistant and icy road surfaces includes: acquiring road surface image sequences, surface temperature sequences, ambient temperature and humidity sequences, and precipitation records for the same road segment during consecutive low-temperature events, and forming multi-source time-series samples according to the acquisition time; inputting the road surface image sequences into a visual depth network to extract spatial features of road texture, reflective areas, frost and snow particles, and rut water boundaries; inputting the surface temperature sequences, ambient temperature and humidity sequences, and precipitation records into a time-series depth network to extract temporal features of low-temperature duration, warming intervals, and humidity abrupt changes; performing conditional attention fusion on the spatial features and temporal features to obtain ice-related features; and applying sequence constraints to the state probabilities of the ice-related features based on a preset ice phase transition relationship to output the road surface condition assessment result.

[0008] Preferably, when forming the multi-source time-series sample, the moment when the road surface temperature changes from above freezing to below freezing, the moment when the precipitation state changes from dry to wet, and the moment when the continuous area of ​​bright spots in the road surface image changes abruptly are used as segmented anchor points for low-temperature events. The pre-state, phase transition state, and recovery state images are extracted according to the segmented anchor points, and the road surface temperature change, cumulative duration below zero, continuous humidity increment, and precipitation duration between adjacent image frames are written into the corresponding samples, so that the formed sample simultaneously contains the appearance change chain, the temperature and humidity evolution chain, and the moisture supply chain.

[0009] Preferably, when extracting the spatial features, the road surface image sequence is first separated into road surface regions, retaining the anti-icing pavement surface and shielding the sky, vehicles, and roadside background; then, a multi-scale convolutional layer is used to extract the rough aggregate texture, continuous reflection of thin ice, frost particle edges, snow compaction marks, and rut water film contours; and a channel attention layer is used to calculate the response weights of different scale features to dry, wet, supercooled water film, nascent thin ice, stable ice film, and melting recovery state, and the weighted multi-scale features are encoded into frame-level visual embeddings.

[0010] Preferably, when extracting the temporal features and performing the conditional attention fusion, the road surface temperature sequence is divided into a cooling segment, a low-temperature maintenance segment, and a warming segment; the environmental temperature and humidity sequence and the precipitation state record are encoded into an environmental driving vector; the environmental driving vector is used as a query term, and the spatial features are used as key terms to calculate the ice-related weights of each image frame under different temperature and humidity stages; then the ice-related weights are concatenated with the spatial features of adjacent frames and temporally convolutionally encoded to obtain a fusion sequence for state probability calculation.

[0011] Preferably, when aligning the appearance change chain, the temperature and humidity evolution chain, and the moisture supply chain, the bright spot expansion start point, the gray-white coverage start point, and the rut residual water shrinkage start point are extracted from the road surface image sequence, respectively, and the freezing point crossing point, the lowest temperature holding interval, and the warming crossing point are extracted from the road surface temperature sequence; anti-icing hysteresis markers are generated based on the time offset between the two types of start points, and the anti-icing hysteresis markers are used as additional inputs to the temporal depth network, participating in temporal feature encoding together with the cumulative duration below zero degrees.

[0012] Preferably, when generating the frame-level visual embedding, image frames with similar appearances in low-temperature humidity, meltwater residue, nascent thin ice, and stable ice film are grouped into a difficult-to-distinguish sample group; the continuity features of local reflective boundaries, the discreteness features of frost and snow particles, the proportion of aggregate texture covered, and the contraction direction features of rut water boundaries are calculated for the difficult-to-distinguish sample group respectively; and when training the visual depth network, the convergence of visual embeddings of the same ice phase state to the state center and the separation of visual embeddings with similar appearances but different ice phases in the state content space are used as training constraints.

[0013] Preferably, the visual depth network includes a texture stabilization branch and a phase transition boundary branch. The texture stabilization branch receives a dry road surface reference frame before the start of a low-temperature event and extracts a fixed texture base of the anti-icing paved surface. The phase transition boundary branch receives the current image frame and extracts the boundaries of water film, ice film, frost layer, and snow cover. The fixed texture base and the coverage boundaries are differentially encoded, and the differential encoding result is residually fused with the multi-scale features and then input into the channel attention layer to generate the frame-level visual embedding.

[0014] Preferably, when constraining the state probabilities of the ice-related features, a directed transition graph is established, including dryness, wetness, supercooled water film, nascent thin ice, stable ice film, and ice melt recovery. According to the cooling segment, the low-temperature maintenance segment, and the warming segment, transition edges matching the current temperature and humidity stage are opened respectively, and the adjacent state probabilities corresponding to unopened transition edges are set as restricted probabilities. Path decoding is performed on the state probabilities of consecutive image frames, and the intermediate states that repeatedly jump in the decoding path are merged into a single state node to obtain a state sequence that satisfies the directed transition graph.

[0015] Preferably, during the path decoding process of the state sequence, an anti-freezing hysteresis marker is generated based on the time offset between the image change start point and the temperature change start point within the same low-temperature event; when there is no open transition edge between the candidate state generated by the fusion sequence and the previous state of the state sequence, the local feature position of the candidate state in the corresponding image frame is retained, and the candidate state is rewritten as an intermediate state to be determined; the intermediate state to be determined and the anti-freezing hysteresis marker of the subsequent image frame are input into the temporal depth network to recalculate the state probability of the corresponding frame.

[0016] Preferably, before outputting the road surface condition assessment result, the state probability, the intermediate state to be determined, and the state sequence are subjected to multi-task consistency processing; the current road surface condition is generated from the current frame state probability, the short-term icing risk level is generated from the continuous frame state sequence, and the anti-icing effect lag degree is generated from the anti-icing lag marker and the number of continuous frames of the intermediate state to be determined; when the timestamps corresponding to the current road surface condition, the short-term icing risk level, and the anti-icing effect lag degree are inconsistent, the three types of results are rearranged and output using the path timestamp of the state sequence.

[0017] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. This invention forms multi-source time-series samples by recording road surface image sequences, road surface temperature sequences, ambient temperature and humidity sequences, and precipitation status within consecutive low-temperature events on the same road segment. Spatial features such as road texture, reflective areas, frost and snow particles, and rut water boundaries are extracted using a visual depth network. Temporal features such as low-temperature duration, warming intervals, humidity abrupt changes, and moisture supply processes are extracted using a temporal depth network. Then, temperature and humidity-driven features are fused with image appearance features using a conditional attention approach. The fused ice-related features are also constrained by ice phase transition relationships such as dryness, wetness, supercooled water film, nascent thin ice, stable ice film, and ice melt recovery. Therefore, state assessment no longer relies solely on a single frame appearance or a single environmental threshold, but is simultaneously limited by visible changes in the road surface, low-temperature accumulation conditions, and the ice phase evolution sequence. This reduces confusion between thin ice, wet reflections, ice melt residue, and frost layers, ensuring that the output state is consistent with the actual phase transition process in consecutive low-temperature events.

[0018] 2. This invention further defines the evaluation process through segmented anchor points, anti-icing hysteresis markers, difficult-to-distinguish sample groups, stable ripple branches, phase transition boundary branches, and directed transition diagrams. Segmented anchor points divide low-temperature events into pre-state, phase transition, and recovery states, ensuring that samples include appearance change chains, temperature and humidity evolution chains, and moisture supply chains. Anti-icing hysteresis markers incorporate the time offset between the image change start point and the temperature change start point into temporal coding. Difficult-to-distinguish sample groups differentiate low-temperature wetting, meltwater residue, nascent thin ice, and stable ice film within the state content space. Stable ripple branches and phase transition boundary branches reduce pavement background color interference through differential coding of the dry reference frame and the current coverage boundary. Directed transition diagrams restrict state paths that do not conform to the low-temperature and warming stages. These techniques collectively enable the current state, short-term icing risk level, and the degree of hysteresis hysteresis effect to be generated within the same time series. Attached Figure Description

[0019] Figure 1 This is an overall flowchart of the deep learning-based method for assessing the condition of cold-resistant and ice-resistant pavement in this invention. Figure 2 This is a flowchart of the multi-source time-series sample construction and low-temperature event segmentation process of the present invention; Figure 3 This is a flowchart of the visual feature extraction and conditional attention fusion process of the present invention; Figure 4 This is a flowchart of the ice phase transfer constraint and multi-task consistent output of the present invention. Detailed Implementation

[0020] refer to Figure 1In one embodiment, the deep learning-based method for assessing the condition of cold-resistant and icy road surfaces uses continuous low-temperature events as processing boundaries. It unifies road surface image sequences, surface temperature sequences, ambient temperature and humidity sequences, and precipitation records obtained within the same low-temperature event on the same road segment onto the acquisition timeline, forming multi-source time-series samples. The road surface image sequences are input into a visual deep network to extract spatial features of road texture, reflective areas, frost and snow particles, and rut water boundaries. The surface temperature sequences, ambient temperature and humidity sequences, and precipitation records are input into a temporal deep network to extract temporal features of low-temperature duration, warming intervals, and humidity abrupt changes. Finally, a conditional attention fusion process is used to integrate these temporal features into the spatial features of the images. The selection and weighting of inter-features assign different ice-related weights to the image appearances in the low-temperature, wet, and warm-up stages. Then, based on the preset ice phase transition relationship between dryness, wetness, supercooled water film, nascent thin ice, stable ice film, and ice melt recovery, the obtained state probabilities after fusion are sequentially constrained, and the road surface state assessment results are output. The working principle of this embodiment is to put the visible appearance, low-temperature accumulation, and ice phase evolution path into the same inference chain, so that thin ice, wet reflection, ice melt residue, and frost layer are no longer judged solely based on single-frame grayscale or brightness differences. The advantage of this embodiment is that the assessment results have state consistency and phase transition traceability within continuous low-temperature events.

[0021] Multi-source time series samples can be represented as follows:

[0022] Wherein, it represents the first Multi-source time-series samples at each acquisition time. Indicates the first Road surface images at each acquisition time, Indicates the first Road surface temperature at each sampling time, Indicates the first An environmental temperature and humidity vector at each acquisition time. Indicates air temperature. Indicates relative humidity. This indicates a record of precipitation conditions. This represents the data acquisition timestamp, which physically signifies a synchronous description of the visible road surface state, thermal state, environmental hydrothermal state, and moisture supply state at the same moment. For example, if the road surface image acquired at the 8th acquisition time is... The road surface temperature is -2.4°C, the air temperature is -3.1°C, the relative humidity is 86%, the precipitation status is recorded as "with water," and the timestamp is the 32nd minute. ,in This indicates the presence of precipitation or road surface water replenishment records, and the calculation results form synchronous sample tuples that can be directly input into subsequent deep learning processing.

[0023] In this embodiment, the multi-source time series sample fields and their processing meanings are shown in Table 1.

[0024] Table 1. Fields and Processing Meanings of Multi-Source Time Series Samples

[0025] refer to Figure 2 In one embodiment, when forming multi-source time-series samples, continuous low-temperature events are used as the upper-level constraints of the sample window. The start and end of the low-temperature events are not determined by a single temperature point, but are provided by the moments when the road surface temperature changes from above freezing to below freezing, the moments when the precipitation state changes from dry to wet, and the moments when the continuous area of ​​bright spots in the road surface image changes abruptly. During data processing, the image frames between adjacent anchor points are divided into the pre-state, phase transition state, and recovery state. The pre-state is used to record the texture reference when the anti-icing pavement surface is not covered by water film or ice film. The phase transition state is used to record the gradual formation of water film, frost layer, and thin ice. The resulting appearance changes and recovery states are used to record the process of water film shrinkage and texture re-emergence after warming or anti-icing effects. Within each segment, the amount of road surface temperature change, cumulative duration below zero, continuous humidity increment, and precipitation duration between adjacent image frames are written into the corresponding sample, resulting in three synchronous links: appearance change chain, temperature and humidity evolution chain, and moisture supply chain. In this embodiment, the fields in Table 1 are not used as isolated inputs, but are bound to the same state evolution sample through timestamps. The advantage of this embodiment is that the sample can retain continuous information from environmental changes to road surface phase changes and then to the recovery process.

[0026] The cumulative duration below zero degrees Celsius and the amount of cooling capacity required can be calculated as follows:

[0027] in, Indicates the first The cumulative low-temperature driving quantity corresponding to each acquisition time. This indicates the cumulative historical data collection time. This indicates the total number of data collection points contained within the cumulative window. Indicates the freezing point reference temperature. Indicates the first Road surface temperature at each sampling time, Indicates the time interval between adjacent data collection times. This means that cooling is only counted when the road surface temperature is below the freezing point reference. Physically, it represents the cumulative effect of low temperatures on the road surface over a period of time. The cumulative road surface temperatures at the four collection points within the window were 0.6, -1.2, -2.0, and -0.7, respectively. minutes, then The calculation results indicate that there is a cumulative low-temperature drive on the order of 3.9 minutes of temperature in the current window.

[0028] refer to Figure 3 In one embodiment, before receiving the road surface image sequence, the visual depth network performs road surface region separation on each frame of the image. The road surface region separation process can use a trained semantic segmentation network to distinguish anti-icing pavement surfaces, sky, vehicles, roadside background, marking shadows, and edge facilities at the pixel level, and retain only the mask area corresponding to the anti-icing pavement surface. The mask area is multiplied with the original image to form an effective image for visual feature extraction. Subsequently, a multi-scale convolutional layer is used to extract rough aggregate texture, continuous reflection of thin ice, frost particle edges, snow compaction marks, and rut water film contours. The shallow features in the multi-scale convolutional layer are used to retain local bright spots and particle edges, the medium-scale features are used to describe the continuous boundaries of water film and thin ice, and the deep features are used to express the overall shape of the covered area. Then, the channel attention layer calculates the response weights of different scale features to dry, wet, supercooled water film, nascent thin ice, stable ice film, and melting recovery state, and encodes the weighted multi-scale features into frame-level visual embeddings. The advantage of this embodiment is that the visual features are no longer excessively affected by non-road surface backgrounds and fixed pavement background colors.

[0029] Frame-level visual embedding can be calculated as follows:

[0030] in, Indicates the first Frame-level visual embedding at each acquisition moment This represents the total number of multi-scale feature layers. Indicates the first One scale, Indicates the first The first data collection time Channel attention weights for each scale feature. Indicates the first Deep feature extraction mapping at various scales, Indicates the first The road surface area mask at each data acquisition time. This represents a pixel multiplication operation at the same location. Physically, it means contributing to the road surface appearance representation at different scales after masking non-road areas. If an image frame contains features at three scales... , , The scalar responses after mapping at the three scales are 2, 5, and 4, respectively. Therefore, the visual embedding value of this frame in the corresponding feature dimension is 0.25×2+0.50×5+0.25×4=4.0.

[0031] In one embodiment, after receiving the road surface temperature sequence, ambient temperature and humidity sequence, and precipitation status record, the temporal deep network divides the road surface temperature sequence into a cooling segment, a low-temperature maintenance segment, and a warming segment. The ambient temperature and humidity sequence and precipitation status record are encoded into an environmental driving vector. The environmental driving vector includes the road surface temperature change, the cumulative low-temperature driving amount, the continuous humidity increment, the precipitation persistence marker, and the warming interval. Since the same bright spot may correspond to a supercooled water film in the cooling segment, to nascent thin ice in the low-temperature maintenance segment, and to meltwater residue in the warming segment, the environmental driving vector is used as the query term and the spatial features are used as the key term during conditional attention fusion. The ice-related weights of each image frame under different temperature and humidity stages are calculated. Then, the ice-related weights are concatenated with the spatial features of adjacent frames and temporally convolutionally encoded to form a fusion sequence for state probability calculation. The advantage of this embodiment is that temperature and humidity changes are not just additional variables, but participate in selecting which local appearances in the image should be interpreted as ice-related features.

[0032] Conditional attention weights can be calculated as follows:

[0033] in, Indicates the first The first data collection time Ice-related weights for local visual features, Indicates the first The environment driving vector at each acquisition time. Indicates the first The first data collection time A local visual embedding, The query mapping matrix represents the environment-driven vector. The key mapping matrix representing visual embedding, This represents the dimension of the mapped vector. This indicates the number of local visual features involved in the calculation within the current image frame. Indicates exponentiation. This represents the transpose operation, which physically means selecting a local road surface region more likely to be associated with icing under given temperature and humidity conditions. If two local visual features exist at a certain moment, their similarity after mapping is 1.0 and 0.0 respectively. ,but , .

[0034] In this embodiment, the ice phase state, directional transition relationship, and temperature and humidity stage constraints are shown in Table 2.

[0035] Table 2. Ice phase states, directional transition relationships, and constraints of temperature and humidity stages.

[0036] refer to Figure 4 In one embodiment, when constraining the state probabilities of ice-related features using sequences, a directed transition graph is established based on Table 2. The nodes in the graph represent dryness, wetness, supercooled water film, nascent thin ice, stable ice film, and ice melt recovery. The edges in the graph represent allowed adjacent state transition relationships. The state probabilities are generated after the fused sequence enters the classification mapping. The classification mapping can use a normalized exponential function to map the probability values ​​of each state to each acquisition time. During path decoding, the state with the highest probability in a single frame is not directly selected; instead, both the probability of a single frame and the probability of adjacent frames are considered simultaneously throughout the entire continuous low-temperature event. The transition principle is as follows: during the cooling phase, only transition edges from dry to moist, from moist to supercooled water film, from supercooled water film to nascent thin ice, and from nascent thin ice to stable ice film are open. During the low-temperature maintenance phase, the open transition edges are maintained and related to the deepening of ice condensation. During the warming phase, the transition edges from stable ice film or nascent thin ice to ice melting recovery and from ice melting recovery to moist or dry are open. The probability of adjacent states corresponding to unopened transition edges is set as a restricted probability, and path decoding is performed on continuous image frames. The advantage of this embodiment is that adjacent sampling results are limited by the phase transition path, reducing jumps that are inconsistent with the low-temperature event stage.

[0037] The state sequence path can be decoded in the following way:

[0038] in, This represents the optimal state path obtained through decoding. Indicates the first Candidate states at each acquisition time. This represents the total number of data collection moments within a continuous low-temperature event. Indicates the first Candidate states at each acquisition time The deep learning output probability, Indicates the transfer constraint weight. Indicates from the previous state To the current state The directed transition open value is 1 when open and 0 when closed. This refers to preventing undefined small positive numbers in logarithmic operations. Physically, it means suppressing paths that do not conform to the ice facies evolution sequence while ensuring the reliability of single-frame state probabilities. If, at a certain adjacent time step, the single-frame probability combination from wet to stable ice film is high, but the corresponding transition is not open, then... If a significant penalty is incurred, path decoding will prioritize the continuous path from wetting to supercooled water film to nascent thin ice.

[0039] In a preferred embodiment, when aligning the appearance change chain, temperature and humidity evolution chain, and moisture supply chain, the starting point of bright spot expansion, the starting point of gray-white coverage, and the starting point of rut water shrinkage are extracted from the road surface image sequence. The starting point of bright spot expansion can be determined by the moment when the increase in the area of ​​the reflective region in a continuous image exceeds the average fluctuation of its preceding and following neighborhoods. The starting point of gray-white coverage can be determined by the moment when the response of frost and snow particles changes from discrete points to continuous patches. The starting point of rut water shrinkage can be determined by the moment when the water film boundary moves towards the rut centerline and the boundary length continuously shortens. Simultaneously, the freezing point crossing point and the lowest temperature holding point are extracted from the road surface temperature sequence. The interval and the temperature recovery crossing point are defined as follows: the freezing point crossing point is the moment when the road surface temperature changes from non-negative to negative and remains at a low temperature; the lowest temperature holding interval is the interval where the temperature change range is relatively small for multiple consecutive acquisition times; and the temperature recovery crossing point is the moment when the road surface temperature changes from negative to non-negative and continues to rise. Then, an anti-icing hysteresis marker is generated based on the time offset between the image change starting point and the temperature change starting point, and the anti-icing hysteresis marker is used as an additional input to the temporal deep network. The advantage of this embodiment is that the phase transition delay caused by anti-icing paving is encoded into the temporal features and is not misjudged as acquisition error or single frame anomaly.

[0040] The anti-freezing hysteresis marker can be calculated as follows:

[0041] in, Indicates the first Anti-freezing hysteresis markers for consecutive low-temperature events This indicates the time corresponding to the starting point of the image change during the low-temperature event. This indicates the time corresponding to the point where the road surface temperature crosses the freezing point during this low-temperature event. Physically, it represents the time delay of the visible phase change of the road surface relative to the moment the low temperature was triggered. If the road surface temperature crosses the freezing point at 12 minutes, and the bright spot expansion in the image appears at 18 minutes, then... The calculation results indicate that in this cryogenic event, the visible phase transition lags behind the temperature trigger by 6 minutes.

[0042] In a preferred embodiment, to address the issue of similar appearances among low-temperature humidity, meltwater residue, newly formed thin ice, and stable ice film, the visual depth network is trained by grouping image frames corresponding to these states into a difficult-to-distinguish sample group. This difficult-to-distinguish sample group is not selected based on simple brightness or grayscale similarity, but rather by selecting image frames with similar reflective or gray-white coverage but different artificial labeling within the same road segment, under similar temperature and humidity conditions, and during adjacent low-temperature events. Subsequently, the continuity characteristics of local reflective boundaries, the dispersion characteristics of frost and snow particles, the proportion of aggregate texture covered, and the contraction direction characteristics of rut water boundaries are calculated for the difficult-to-distinguish sample group. The continuity feature can be determined by the connectivity length and number of breaks of the boundary pixels of the reflective area. The discreteness feature of frost and snow particles can be determined by the number of connected domains and spacing distribution of particles. The proportion of aggregate texture covered can be determined by the matching residual between the current image and the dry reference texture. The contraction direction feature of the rut water boundary can be determined by the normal displacement of the water film edge of adjacent frames. In the training constraints, the visual embeddings of the same ice phase state converge towards the corresponding state center, and the visual embeddings with similar appearances but different ice phases are separated in the state content space. The advantage of this embodiment is that the model training process can take the phase transition differences in similar appearance samples as the learning focus.

[0043] Preferably, the visual depth network can be configured with a texture stabilization branch and a phase transition boundary branch. The texture stabilization branch receives a dry road surface reference frame before the start of a low-temperature event. The dry road surface reference frame can be selected from historical samples where the road surface temperature is above freezing, the precipitation state is waterless, and there is no continuous reflection or gray-white coverage in the image. The texture stabilization branch extracts a fixed texture base of the anti-icing pavement surface from the reference frame, including aggregate particle size, pavement color distribution, and dry texture direction. The phase transition boundary branch receives the current image frame and extracts the boundaries of water film, ice film, frost layer, and snow cover. After the output of the two branches, differential encoding is performed. The differential encoding is used to describe the occlusion position, occlusion ratio, and boundary change direction of the current coverage boundary on the fixed texture base. The differential encoding result and multi-scale features are fused through residual method and then enter the channel attention layer. The channel attention layer redistributes the response weights of each scale feature according to the differential texture occlusion information. The advantage of this embodiment is that the fixed texture and phase transition coverage are separated, and the interference of road surface color difference and long-term wear texture on state recognition is reduced.

[0044] In one embodiment, the openness relationships of the directed transition graph change with the temperature and humidity stages. During the cooling phase, if the environmental driving vector indicates that the temperature is decreasing and the precipitation state is recorded as wet, then transition edges from dry to wet, from wet to supercooled water film, and from supercooled water film to nascent thin ice are opened, while transitions from stable ice film directly to dry are restricted. During the low-temperature maintenance phase, if the cumulative low-temperature driving force continues to increase and the humidity remains high, then transition edges from nascent thin ice to stable ice film and the maintenance of various icing states are opened, while transitions from melting ice back to nascent thin ice are restricted. During the warming phase, if the road surface temperature continues to rise… Furthermore, as the boundary of residual water in the ruts shrinks, transition edges are opened from stable ice film to ice melt recovery, from nascent thin ice to ice melt recovery, and from ice melt recovery to wet or dry. After path decoding, the intermediate states of repeated transitions are merged. During merging, intermediate nodes with short consecutive occurrence times and identical states before and after are taken as repeated transition nodes, and their states are incorporated into the consecutive states before and after. If the intermediate node is located near the temperature inflection point or the starting point of image change, it is retained as an intermediate state to be judged. The advantage of this embodiment is that the state sequence retains the real phase transition boundary and avoids meaningless repeated transitions caused by single-frame probability fluctuations.

[0045] Furthermore, during the state sequence path decoding process, when there is no open transition edge between the candidate state generated by the fusion sequence and the previous state of the state sequence, the candidate state is not directly discarded. Instead, the local feature position of the candidate state in the corresponding image frame is retained. The local feature position includes the reflective boundary region, gray-white coverage region, rut water film region, or frost particle aggregation region that triggers the candidate state. The candidate state is then rewritten as an intermediate state to be judged. The intermediate state to be judged and the anti-icing hysteresis marker of the subsequent image frame are input into the temporal depth network for secondary calculation. During the secondary calculation, the change direction of the same local region in the subsequent several frames is used to determine whether the candidate state belongs to the true phase transition starting point. If the local reflective boundary continues to expand and the low temperature cumulative driving amount increases, the icing direction candidate is maintained. If the local reflective boundary contracts and the warming segment has been formed, it is rewritten as the melting recovery or moist state. If the local feature only appears in a single frame and there is no continuous position corresponding in adjacent frames, it is treated as an isolated visual perturbation. The advantage of this embodiment is that abnormal candidates will not destroy the entire state path, and the local true phase transition starting point can be retained and re-judged.

[0046] In one embodiment, before outputting the pavement condition assessment results, the state probability, the intermediate state to be determined, and the state sequence are processed for multi-task consistency. The current pavement condition is determined by the current frame state probability and the path decoding state. The short-term icing risk level is determined by the continuous frame state sequence, the cumulative low temperature driving amount, and the precipitation duration marker. The anti-icing effect lag is determined by the anti-icing lag marker and the number of continuous frames of the intermediate state to be determined. The three types of results are output on the same time axis. When the timestamp corresponding to the current pavement condition, the timestamp corresponding to the short-term icing risk level, and the timestamp corresponding to the anti-icing effect lag are inconsistent, they are rearranged based on the path timestamp of the state sequence so that the three types of results point to the same collection time or the same low temperature event stage. The output record includes at least the collection time, pavement condition, state probability, icing risk level, anti-icing lag marker, and the location of the main local features used for determination. The advantage of this embodiment is that the current state, risk level, and lag are based on the same state sequence, avoiding conflicting results at the same sampling time.

[0047] In a preferred embodiment, the deep learning training process uses labeled low-temperature event samples and incompletely labeled continuous samples to jointly construct a training set. The labeled samples are used to provide state supervision for dryness, wetness, supercooled water film, nascent thin ice, stable ice film, and ice melt recovery. The incompletely labeled continuous samples are used to provide state transition order and temperature and humidity stage changes. During training, the visual deep network first learns the road surface area mask, fixed texture base, phase transition boundary, and frame-level visual embedding. The temporal deep network simultaneously learns the cumulative driving amount of low temperature, continuous humidity increment, precipitation continuous label, and anti-icing hysteresis label. The conditional attention fusion process receives continuous frames of the same low-temperature event in each training batch instead of random single frames. The state sequence constraint penalizes the predicted path that does not conform to the open transition edge during the training phase and retains the local position record of the intermediate state to be judged. After training, the visual feature map, temporal feature map, conditional attention weight calculation relationship, and directed transition graph open rules are saved. The advantage of this embodiment is that the trained model simultaneously learns the differences in road surface appearance, low-temperature driving conditions, and phase transition path constraints.

[0048] In one embodiment, after receiving continuous low-temperature event data of the road segment to be evaluated during the inference phase, a sliding event window is constructed according to the collection timestamp. Each window contains several image frames, road surface temperature records, environmental temperature and humidity records, and precipitation status records prior to the current collection time. The visual depth network extracts frame-level visual embeddings for each frame within the window, and the temporal depth network extracts environmental driving vectors from the temperature and humidity records within the window. The conditional attention fusion process generates ice-related features for each frame, and the state probability mapping gives the probability distribution of each frame for dryness, wetness, supercooled water film, nascent thin ice, stable ice film, and ice melt recovery. The directed transition graph opens corresponding transition edges according to the cooling segment, low-temperature maintenance segment, or warming segment to which the current window belongs. The path decoding obtains the optimal state sequence within the window. If the front segment of the window overlaps with the back segment of the previous window, the state with higher path consistency in the overlapping area is used as the connecting state. If there is a conflict in the connecting states, the path with higher probability and open transition relationship is retained. The advantage of this embodiment is that it can maintain the state continuity relationship between adjacent windows during continuous operation.

[0049] In a preferred embodiment, when the road surface area separation result shows occlusion or partial loss, the frame-level visual embedding is processed to handle the missing region. The occluded region is determined by the difference in masking between consecutive frames and the discontinuity of local texture. If the occluded region is located in a non-rut area and does not cover the main reflective boundary, the weight of the region is reduced in the visual embedding. If the occluded region covers the boundary of rut water or the continuous reflective boundary of thin ice, the corresponding local feature of the frame is marked as an incomplete feature and compensated by the adjacent frame. When the adjacent frame is compensated, the boundary movement direction and temperature and humidity stage of the same image coordinate area in the previous and next frames are consistent as the matching condition. After successful matching, the incomplete feature is replaced by the weighted average of the local features of the previous and next frames. When the matching fails, the corresponding local feature is removed from the conditional attention calculation but the temporal feature is retained. The advantage of this embodiment is that short-term occlusion or partial loss will not be mistakenly taken as the input state path of ice phase change.

[0050] In one embodiment, the training constraints for difficult-to-distinguish sample groups can also be used in conjunction with directed transition graphs. If image frames with similar appearances but different state labels appear within a training batch, the allowed adjacent state relationships are determined based on their low-temperature event stage. Sample groups in the cooling phase with increasing cumulative low-temperature driving force are allowed to form a continuous transition between wetting, supercooled water film, and nascent thin ice. Sample groups in the warming phase with shrinking water film boundaries are allowed to form a continuous transition between stable ice film, melting recovery, and wetting. When the visual embedding separates in the state content space, it still maintains that the distance between adjacent phase transition states is smaller than the distance between non-adjacent states, avoiding misclassifying real continuous phase transitions into unrelated categories. At the same time, it maintains a large content interval for non-adjacent states such as dryness and stable ice film. The advantage of this embodiment is that the classification boundary is consistent with the ice phase evolution order, and the deep learning training results can be directly used by subsequent path decoding.

[0051] In one embodiment, when the anti-icing hysteresis marker participates in the temporal deep network encoding, positive hysteresis, zero hysteresis, and negative hysteresis can be converted into different time offset features. Positive hysteresis indicates that the image phase change is later than the road surface temperature crossing the freezing point, zero hysteresis indicates that the image phase change is basically synchronous with the road surface temperature crossing the freezing point, and negative hysteresis indicates that the image change precedes the crossing of the freezing point and usually corresponds to precipitation replenishment or snowmelt residual water changes. After receiving the time offset features, the temporal deep network combines the low temperature cumulative driving amount and precipitation persistence marker to generate hysteresis-related temporal embeddings. If positive hysteresis persists and the candidate state gradually enters the supercooled water film from wet, the embedding supports the state path of delayed freezing due to anti-icing effect. If negative hysteresis occurs and the temperature has not yet fallen below the freezing point, the embedding restricts direct entry into the initial thin ice or stable ice film state. The advantage of this embodiment is that the phase change hysteresis caused by anti-icing material is distinguished from ordinary wet changes in the temporal features.

[0052] In a preferred embodiment, the generation of the short-term icing risk level is not independent of the current road surface state, but is jointly deduced by the subsequent change trend of the state sequence, the cumulative driving force of low temperature, and the moisture supply chain. If the current state is wet and the cumulative driving force of low temperature increases continuously, and the precipitation state record retains water, then the candidate path for the transfer from wetness to supercooled water film or nascent thin ice is searched in the state sequence. If the current state is ice melting recovery and the warming crossing point has appeared, then the selection priority of the candidate path in the direction of icing is reduced. If the current state is supercooled water film and the bright spot boundary continues to expand, then the candidate path for the transfer to nascent thin ice is retained. The risk level record does not change the current state label, but is only used as the short-term path branch result under the same state sequence. The lag degree of anti-icing effect is generated based on the anti-icing lag mark and the number of frames of the intermediate state to be judged. The advantage of this embodiment is that the risk level, the current state, and the lag degree have a logical correspondence within the same event window.

[0053] In one embodiment, the output record can be used for subsequent verification without relying on the internal calculation process of the original model. The record saves the road surface state, state probability, open transfer edges, restricted transfer edges, main local feature locations, anti-icing hysteresis markers, and low temperature cumulative driving amount for each acquisition moment. When verifying a certain state result, the record can be used to reconstruct which local image features, which temperature and humidity stage features, and which directed transfer relationships jointly formed the state. If the single frame probability at a certain acquisition moment is inconsistent with the state path, the record retains the source of the restricted probability and the intermediate state processing result to be judged, which makes it easy to determine whether the difference comes from single frame visual disturbance, temperature and humidity stage restriction, or retention of local phase change starting point. The advantage of this embodiment is that the state evaluation result has a traceable data chain, which facilitates technical verification of boundary state, phase change state, and recovery state.

Claims

1. A deep learning-based method for assessing the condition of cold-resistant and icy road surfaces, characterized in that, include: Acquire road surface image sequences, road surface temperature sequences, ambient temperature and humidity sequences, and precipitation status records for the same road segment during consecutive low-temperature events, and form multi-source time-series samples according to the acquisition time; The road surface image sequence is input into a visual depth network to extract spatial features of road surface texture, reflective areas, frost and snow particles, and rut water boundaries. The road surface temperature sequence, the ambient temperature and humidity sequence, and the precipitation status record are input into a time-series deep network to extract the time-series features of persistent low temperature, warming interval, and abrupt humidity changes. Conditional attention fusion is performed on the spatial features and the temporal features to obtain ice-related features; Based on the preset ice phase transition relationship, the state probabilities of the ice-related features are sequentially constrained, and the road surface state assessment results are output.

2. The deep learning-based method for assessing the condition of cold-resistant and icy road surfaces according to claim 1, characterized in that, When forming the multi-source time series samples, the moments when the road surface temperature changes from above freezing to below freezing, the moments when the precipitation state changes from dry to wet, and the moments when the continuous area of ​​bright spots in the road surface image changes abruptly are used as segmented anchor points for low-temperature events. Images of the pre-state, phase transition state, and recovery state are extracted according to the segmented anchor points. The changes in road surface temperature, cumulative duration below zero degrees, continuous increase in humidity, and continuous precipitation between adjacent image frames are written into the corresponding samples, so that the formed samples simultaneously contain the appearance change chain, temperature and humidity evolution chain, and moisture supply chain.

3. The deep learning-based method for assessing the condition of cold-resistant and icy road surfaces according to claim 1, characterized in that, When extracting the spatial features, the road surface image sequence is first separated into road surface regions, the anti-icing pavement surface is preserved and the sky, vehicles and roadside background are shielded; Then, multi-scale convolutional layers are used to extract the texture of rough aggregate, continuous reflection of thin ice, edge of frost particles, snow compaction marks and outline of rut water film; The channel attention layer is used to calculate the response weights of features at different scales to dry, wet, supercooled water film, nascent thin ice, stable ice film and ice melt recovery state, and the weighted multi-scale features are encoded into frame-level visual embeddings.

4. The deep learning-based method for assessing the condition of cold-resistant and icy road surfaces according to claim 1, characterized in that, When extracting the temporal features and performing the conditional attention fusion, the road surface temperature sequence is divided into a cooling segment, a low temperature maintenance segment, and a warming segment, and the environmental temperature and humidity sequence and the precipitation status record are encoded into an environmental driving vector. Using the environmental driving vector as the query term and the spatial features as the key term, calculate the ice-related weights of each image frame under different temperature and humidity stages. The ice-associated weights are then concatenated with the spatial features of adjacent frames and temporally convolutionally encoded to obtain a fusion sequence for state probability calculation.

5. The deep learning-based method for assessing the condition of cold-resistant and icy road surfaces according to claim 2, characterized in that, When aligning the appearance change chain, the temperature and humidity evolution chain and the moisture supply chain, the bright spot expansion start point, the gray-white coverage start point and the rut water shrinkage start point are extracted from the road surface image sequence, and the freezing point crossing point, the lowest temperature holding interval and the warming crossing point are extracted from the road surface temperature sequence. Anti-freezing hysteresis markers are generated based on the time offset between the two types of starting points. These anti-freezing hysteresis markers are used as additional inputs to the temporal deep network and participate in temporal feature encoding together with the cumulative duration below zero degrees.

6. The deep learning-based method for assessing the condition of cold-resistant and ice-resistant pavement according to claim 3, characterized in that, When generating the frame-level visual embedding, image frames with similar appearances in low-temperature humidification, melted ice residue, nascent thin ice, and stable ice film are grouped into a difficult-to-distinguish sample group. For the difficult-to-distinguish sample groups, the continuity characteristics of local reflective boundaries, the dispersion characteristics of frost and snow particles, the proportion of aggregate texture covered, and the shrinkage direction characteristics of rut water boundaries are calculated respectively. During the training of the visual depth network, visual embeddings of the same ice phase converge toward the state center, while visual embeddings of similar appearances but different ice phases are separated in the state content space as training constraints.

7. The deep learning-based method for assessing the condition of cold-resistant and icy road surfaces according to claim 3, characterized in that, The visual depth network includes a texture stabilization branch and a phase transition boundary branch. The texture stabilization branch receives a dry road surface reference frame before the start of a low-temperature event and extracts a fixed texture base for the anti-icing pavement surface. The phase transition boundary branch receives the current image frame and extracts the boundaries of water film, ice film, frost layer and snow cover; The fixed texture base and the coverage boundary are differentially encoded, and the differential encoding result is residually fused with the multi-scale features and then input into the channel attention layer to generate the frame-level visual embedding.

8. The deep learning-based method for assessing the condition of cold-resistant and icy road surfaces according to claim 4, characterized in that, When constraining the state probabilities of the ice-related features in sequence, a directed transition graph is established that includes dryness, wetness, supercooled water film, nascent thin ice, stable ice film, and ice melt recovery. According to the cooling segment, the low temperature maintenance segment and the warming segment, respectively open the transfer edge that matches the current temperature and humidity stage, and set the adjacent state probability corresponding to the unopened transfer edge to the restricted probability; Path decoding is performed on the state probabilities of consecutive image frames, and the intermediate states that repeatedly jump in the decoding path are merged into a single state node to obtain a state sequence that satisfies the directed transition graph.

9. The deep learning-based method for assessing the condition of cold-resistant and icy road surfaces according to claim 8, characterized in that, During the path decoding process of the state sequence, an anti-freezing hysteresis marker is generated based on the time offset between the image change start point and the temperature change start point within the same low-temperature event. When there is no open transition edge between the candidate state generated by the fusion sequence and the previous state of the state sequence, the local feature position of the candidate state in the corresponding image frame is retained, and the candidate state is rewritten as an intermediate state to be judged. The intermediate state to be determined and the anti-icing hysteresis label of the subsequent image frame are input together into the temporal depth network to recalculate the state probability of the corresponding frame.

10. The deep learning-based method for assessing the condition of cold-resistant and ice-resistant pavement according to claim 9, characterized in that, Before outputting the road surface state assessment result, the state probability, the intermediate state to be judged, and the state sequence are subjected to multi-task consistency processing. The current road surface state is generated from the probability of the current frame state, the short-term icing risk level is generated from the continuous frame state sequence, and the degree of icing lag is generated from the anti-icing lag marker and the number of continuous frames of the intermediate state to be judged. When the timestamps corresponding to the current road surface condition, the short-term icing risk level, and the degree of lag in the anti-icing effect are inconsistent, the three types of results are rearranged and output using the path timestamps of the state sequence.

Images