A city-level road state perception and early warning method based on vehicle-road cloud multi-source data fusion

Abstract

A kind of city-level road state perception and early warning method based on vehicle-road cloud multi-source data fusion.The present application relates to the technical field of intelligent transportation and vehicle networking road state perception, and aims at the problem that the prior art is difficult to fuse vehicle dynamics data, roadside environment data and meteorological data to realize city-level continuous road surface physical state perception and accurate early warning, and proposes the following scheme: collecting vehicle dynamics data, vehicle position data, roadside environment data and meteorological data, matching the multi-source data to the corresponding road unit and time window, extracting vehicle dynamics indirect perception index, and combining roadside environment data and meteorological data to carry out dynamic confidence weighted fusion, forming structured road surface state data, identifying road surface state and generating early warning information related to vehicle driving route.The present application is suitable for city road surface state perception, driving safety early warning and road maintenance decision support.

Description

Technical Field

[0001] It involves the fields of intelligent transportation and vehicle networking technology, specifically road condition perception and early warning technology. Background Technology

[0002] With the development of vehicle-to-everything (V2X) and intelligent transportation technologies, urban road operation safety places higher demands on road condition perception. Road condition not only includes traffic flow status but also physical conditions that directly affect vehicle driving safety, such as road surface dryness / wetness, water accumulation, icing, slipperiness, and bumps. Among existing road condition perception technologies, one type of solution mainly relies on onboard sensors to collect relevant signals from the vehicle's surroundings or the vehicle itself, such as using data on sound, vibration, and tire-road contact characteristics to determine the road condition at the vehicle's current location. Another type of solution mainly relies on fixed sensors installed on the road surface or roadside to collect local environmental parameters such as road surface temperature, humidity, water accumulation, and friction to determine the road condition near the sensor's location. A third type of solution integrates floating car data, roadside detection data, or meteorological data to estimate road traffic operation status, congestion status, or regional environmental changes, and provides corresponding prompts to vehicles or management systems.

[0003] While the aforementioned solutions can achieve local road condition detection or traffic condition estimation to some extent, they still have shortcomings in urban-level road surface physical condition perception scenarios. For solutions relying solely on single-vehicle data, the perception results mainly reflect the road surface condition at the vehicle's current location, making it difficult to obtain the continuous road surface condition of the road segment ahead or within the urban road network in advance. For solutions relying on fixed roadside sensors, the detection range is limited by the sensor deployment location; to cover the urban road network, a large number of fixed detection devices need to be deployed, resulting in high deployment and maintenance costs and easily creating perception gaps in undeployed areas. Existing multi-source traffic data fusion solutions typically focus on traffic flow parameters such as vehicle speed, flow rate, and density, or on road target detection and traffic event recognition, failing to fully utilize the dynamic response of vehicles during travel as an indirect sensing signal reflecting the physical condition of the road surface.

[0004] In actual road driving, when the road surface changes to conditions such as slipperiness, icing, water accumulation, or bumps, the vehicle's dynamic data, including wheel speed differences, longitudinal acceleration, lateral acceleration, braking pressure, steering response, and ABS activation status, typically change accordingly. While existing technologies can directly detect some road surface parameters using dedicated sensors, they lack sufficient utilization of the correlation between vehicle dynamics data, roadside environmental data, and meteorological data. This makes it difficult to form a mobile sensing network through large-scale vehicle operation and to unify data from different sources, sampling frequencies, and spatial scales into a single road condition assessment process. Therefore, in urban-level road condition perception, problems persist regarding scattered sensing data sources, discontinuous spatial coverage, and insufficient utilization of indirect sensing information from vehicle dynamics.

[0005] Furthermore, existing road condition warning methods mostly employ unified broadcasting or single-notification approaches, typically generating general warning information based on only a single detection point or event. This makes it difficult to combine the vehicle's current location, current route, and the positional relationship of road condition events to filter and distribute warnings to drivers based on their travel path. Simultaneously, for traffic management personnel and road maintenance departments, existing solutions also struggle to leverage continuously accumulated road condition data to create regional situation displays, event statistics, and maintenance decision support. This hinders the collaborative application of road condition perception results between driving safety warnings and road maintenance management.

[0006] In summary, existing technologies have shortcomings in that they cannot unify and integrate vehicle dynamics data, roadside environmental data, and meteorological data to achieve continuous urban road surface physical state perception, and they cannot perform hierarchical personalized early warning distribution and maintenance decision support based on the fused perception results. Summary of the Invention

[0007] To address the shortcomings of existing technologies, such as the difficulty in unifying and integrating vehicle dynamics data, roadside environmental data, and meteorological data to achieve continuous urban-level pavement physical state perception, and the difficulty in providing hierarchical and personalized early warning distribution and maintenance decision support based on the fused perception results, the technical solution provided by this invention is as follows: A city-level road condition perception and early warning method based on vehicle-road-cloud multi-source data fusion includes: The steps involve collecting vehicle dynamics data, vehicle position data, and time information generated by the vehicle during its journey on the road, as well as collecting roadside environmental data and meteorological data, to form multi-source road condition perception data. The step of matching the multi-source road state perception data to the corresponding road unit and the corresponding time window based on vehicle location data, roadside environment data, road area corresponding to meteorological data, and time information to form multi-source matching data for road units. Based on the vehicle dynamics data in the multi-source matching data of the road unit, extract vehicle dynamics indirect perception indicators to characterize changes in road surface conditions. The vehicle dynamics indirect perception indicators include at least the step of indirect perception information formed by wheel speed differences, acceleration anomalies, braking state changes, and steering response changes. The steps involve taking vehicle dynamics indirect perception indicators, roadside environmental data, and meteorological data within the same road unit and the same time window as fusion objects, and determining the fusion weight of each fusion object based on the reliability of data sources, the consistency of multi-vehicle indirect perception information, and the correspondence between meteorological data and roadside environmental data and vehicle dynamics indirect perception indicators. The steps are as follows: weighted fusion of vehicle dynamics indirect perception indicators, roadside environmental data and meteorological data according to the fusion weights to form structured pavement state data for the corresponding road units; The steps for identifying the road condition category and condition confidence level of the corresponding road unit based on the structured road condition data, and forming the road condition identification result; The steps are as follows: generating road condition events based on the road condition recognition results, matching the road condition events with the vehicle's current location and driving route, and forming driver warning information related to the vehicle's driving route.

[0008] Furthermore, in a preferred embodiment, the vehicle dynamics data includes vehicle speed, longitudinal acceleration, lateral acceleration, wheel speeds, braking pressure, and steering angle; the roadside environmental data includes road surface temperature, water depth, icing status, and coefficient of friction; and the meteorological data includes precipitation, air temperature, dew point temperature, and visibility.

[0009] Furthermore, in a preferred embodiment, when matching the multi-source road state perception data to the corresponding road unit and the corresponding time window, the road unit corresponding to the vehicle dynamics data is determined according to the vehicle position data, the road unit corresponding to the roadside environment data is determined according to the road position corresponding to the roadside environment data, the road unit covered by the meteorological data is determined according to the road area corresponding to the meteorological data, and the vehicle dynamics data, roadside environment data and meteorological data in the same time window within the same road unit are combined to form the multi-source matching data of the road unit.

[0010] Furthermore, in a preferred embodiment, the wheel speed difference is determined by the deviation relationship between the wheel speeds of each wheel, the acceleration anomaly is determined by the deviation relationship between longitudinal acceleration and lateral acceleration relative to the historical normal driving state, the braking state change is determined by the brake pressure fluctuation and ABS trigger state, and the steering response change is determined by the correspondence between the steering angle and the lateral acceleration.

[0011] Furthermore, in a preferred embodiment, when determining the fusion weight of each fusion object, when there is valid roadside environmental data in the same road unit, the fusion weight of the roadside environmental data is increased; when the vehicle dynamics indirect perception indicators of multiple vehicles in the same road unit have a consistent abnormal direction, the fusion weight of the vehicle dynamics indirect perception indicators is increased; when meteorological data has a state correspondence with roadside environmental data and vehicle dynamics indirect perception indicators, the fusion confidence of the corresponding road unit is increased.

[0012] Furthermore, in a preferred embodiment, the structured pavement condition data includes road unit identifiers, time windows, vehicle dynamics fusion features, roadside environment fusion features, meteorological fusion features, and comprehensive confidence scores, and serves as the data basis for identifying pavement condition categories and condition confidence scores.

[0013] A city-level road condition perception and early warning device based on vehicle-road-cloud multi-source data fusion includes: The module collects vehicle dynamics data, vehicle position data, and time information generated by the vehicle during its journey on the road, and also collects roadside environmental data and meteorological data to form multi-source road condition perception data. Based on vehicle location data, roadside environment data corresponding to road location, meteorological data corresponding to road area, and time information, the multi-source road state perception data is matched to the corresponding road unit and the corresponding time window to form a module for multi-source matching data of road unit. Based on the vehicle dynamics data in the multi-source matching data of the road unit, vehicle dynamics indirect perception indicators for characterizing changes in road surface condition are extracted. The vehicle dynamics indirect perception indicators include at least a module of indirect perception information formed by wheel speed difference, acceleration anomaly, braking state change and steering response change. The module uses vehicle dynamics indirect perception indicators, roadside environmental data, and meteorological data within the same road unit and the same time window as fusion objects, and determines the fusion weight of each fusion object based on the reliability of data sources, the consistency of multi-vehicle indirect perception information, and the correspondence between meteorological data and roadside environmental data and vehicle dynamics indirect perception indicators. According to the fusion weight, the indirect perception indicators of vehicle dynamics, roadside environmental data and meteorological data are weighted and fused to form a module of structured pavement state data for the corresponding road unit; Based on the structured pavement condition data, a module is formed to identify the pavement condition category and condition confidence level of the corresponding road unit, thus generating a pavement condition identification result. Based on the road surface condition recognition results, a road condition event is generated, and the road condition event is matched with the vehicle's current location and driving route to form a module for driver warning information related to the vehicle's driving route.

[0014] A computer storage medium for storing a computer program, which, when read by the computer, is executed by the computer using the method described thereon.

[0015] A computer, including a processor and a storage medium, executes the method when the processor reads a computer program stored in the storage medium.

[0016] A computer program product, which, as a computer program, implements the method when the computer program is executed.

[0017] Compared with the prior art, the advantages of the technical solution provided by the present invention are as follows: The vehicle-side system collects real-time vehicle dynamics parameters such as vehicle speed, longitudinal acceleration, lateral acceleration, wheel speeds, braking pressure, and steering angle via OBD-II or CAN bus. This transforms road condition perception from relying solely on fixed roadside sensors or a single dedicated road surface sensor into a mobile sensing node. When the road surface becomes slippery, icy, flooded, or bumpy, the vehicle dynamics parameters fluctuate abnormally due to changes in the tire-road adhesion relationship. This compensates for the limited coverage of traditional roadside detection points and provides a denser data source for continuous urban road condition assessment.

[0018] The vehicle-side system uses GPS or BeiDou dual-mode positioning modules to acquire vehicle location information and uploads the vehicle dynamics parameters along with the location information to the cloud. This allows each set of vehicle dynamics anomaly data to be associated with a specific road location. Compared to solutions that only determine the current state of a single vehicle, this feature allows vehicle dynamics anomalies to no longer remain at the vehicle's own control level, but can be mapped to road surface condition cues for the corresponding road segment, thus providing a spatial basis for subsequently forming a road surface condition distribution within the urban road network.

[0019] The roadside sensing layer collects road surface environmental parameters such as road surface temperature, water depth, icing status, and friction coefficient, enabling road condition judgments to obtain direct physical quantity support from fixed detection points. Compared with relying solely on indirect signals from vehicle dynamics, roadside environmental parameters can verify the actual road surface conditions at specific key intersections or road sections, thereby reducing the impact of individual vehicle driving behavior, vehicle type differences, or instantaneous operational differences on the judgment results.

[0020] Meteorological data collection includes regional environmental information such as precipitation, air temperature, dew point temperature, and visibility, enabling road condition assessments to be comprehensively analyzed in conjunction with weather changes. Since road surface conditions such as icing, water accumulation, and slipperiness are typically related to environmental conditions like precipitation, low temperatures, and dew points, meteorological data can provide external corroboration for vehicle dynamics anomalies and roadside measurement data, avoiding direct judgments of road surface hazards based solely on localized anomalies and improving the rationality of urban-level road condition identification.

[0021] The multi-protocol data access gateway receives multi-source data from vehicles, roadside units, and third-party meteorological services, and verifies and filters the data for format, integrity, and anomalies, enabling data from different sources to enter a unified data processing flow. Compared to the scattered use of various sensor data, this feature solves the problem of heterogeneous data sources from vehicles, roads, and meteorological units, providing standardized input for subsequent unified spatiotemporal alignment, fusion calculation, and early warning generation.

[0022] A unified spatial reference is used to spatially align vehicle location data, roadside sensor location data, and road segment information, enabling data from different sources to be attributed to the same road unit or adjacent road areas. This process avoids the difficulty in comparing vehicle dynamics anomalies, roadside detection data, and meteorological data due to their different spatial scales, allowing the system to form fused judgments around specific road segments, rather than generating independent point or line detection results.

[0023] By aligning timestamps from different data sources to a unified benchmark and compensating for differences in sampling frequencies among different sensors through interpolation, high-frequency vehicle dynamics data, low-frequency roadside environmental data, and periodic meteorological data can be matched within the same time window. This feature solves the fusion error problem caused by inconsistent sampling frequencies of multi-source data, enabling vehicle anomalies, low road surface temperatures, precipitation, or water accumulation occurring on a road segment within a similar time frame to be jointly included in the judgment.

[0024] Kalman filtering is used to denoise the sensor data, and outlier removal rules are used to filter out abnormal data, thus controlling the data quality before fusion. Since vehicle dynamics data is easily affected by driving operations, road gradient, vehicle load, and transient communication anomalies, preprocessing can reduce the interference of occasional noise on road condition judgment, making the subsequent fusion results more reflective of continuous and regional road condition changes.

[0025] By extracting wheel speed difference data from CAN bus data, the system can identify road surface adhesion anomalies based on the deviation between the wheel speeds of each wheel. Compared to solutions that directly rely on roadside friction sensors, wheel speed difference data originates from the actual contact process between the vehicle and the road surface. When there is a more obvious difference in slippage between the drive wheels and driven wheels on wet or icy roads, the system can use this anomaly as an indirect basis for sensing low road surface adhesion.

[0026] Extracting acceleration anomalies from longitudinal and lateral acceleration enables the system to identify abnormal vehicle motion responses under similar or identical driving conditions. Low-adhesion surfaces, flooded surfaces, or bumpy road sections alter vehicle acceleration, deceleration, and lateral response. By comparing current acceleration changes with historical normal conditions, potential road anomalies in areas not covered by dedicated roadside sensors can be detected.

[0027] Recording the frequency of ABS triggering and the amplitude of brake pressure fluctuations allows the vehicle safety control intervention during braking to be translated into a basis for judging whether the road surface is slippery or icy. Since frequent ABS triggering is usually related to insufficient tire adhesion during braking, and brake pressure fluctuations can reflect the instability of the braking control process, using both as braking characteristic parameters can more directly characterize the risk of low-adhesion road surfaces than simply changing vehicle speed.

[0028] Monitoring the response relationship between steering angle and lateral acceleration enables the system to identify changes in road adhesion from the deviation between the vehicle's steering input and the actual lateral motion output. When the lateral acceleration response weakens at the same or similar steering angle, it indicates that the tire's lateral force transmission capability may be affected by road conditions such as wetness or iciness. This feature can supplement wheel speed and braking characteristics, forming a more complete indirect perception index system for vehicle dynamics.

[0029] By combining vehicle dynamics characteristics, roadside environmental parameters, and meteorological data as a fusion dimension, road condition judgment no longer relies on a single data source. Vehicle dynamics characteristics reflect actual driving response, roadside environmental parameters provide local direct detection values, and meteorological data provides regional environmental background. After cross-verification of the three, the risk of misjudgment caused by noise affecting a single onboard sensor, insufficient coverage of a single roadside sensor, and insufficient spatial resolution of a single meteorological data source can be reduced.

[0030] By assigning dynamic confidence weights to different data sources based on their reliability, the fusion results can be adjusted according to data quality and applicable scenarios. Roadside sensors have high direct detection value at their respective deployment points, vehicle dynamics indicators have real-time and coverage expansion capabilities, and meteorological data have regional contextual significance. By using dynamic weights rather than fixed averages for fusion, different data sources can play appropriate roles under different road conditions, improving the robustness of the fusion judgment.

[0031] When direct measurements from roadside sensors exist on a road segment, the weight of roadside data is increased, enabling the system to prioritize detection results directly related to the physical condition of the road surface. This approach avoids over-reliance on indirect signals generated by vehicle driving behavior on road segments with existing reliable measured data, thereby improving the reliability of road condition assessments at key intersections or road segments.

[0032] When multiple vehicles generate consistent dynamic anomaly signals on the same road segment, the confidence level of indirect vehicle dynamics indicators is increased, enabling the system to perform cross-verification using a large number of operating vehicles. Compared to single-vehicle anomalies, which may stem from sudden braking by the driver, vehicle model differences, or vehicle malfunctions, consistent anomalies across multiple vehicles are more likely to reflect conditions such as slipperiness, water accumulation, icing, or bumps on the road segment itself. Therefore, this feature can transform scattered vehicle data into an effective basis for judgment in city-level mobile sensing networks.

[0033] When meteorological conditions, abnormal vehicle dynamics signals, and roadside sensor readings are consistent, the overall confidence level is further improved, enabling the system to confirm dangerous road conditions through multi-source collaborative evidence. For example, when low temperatures, abnormal road surface temperatures, and multiple vehicle braking anomalies occur simultaneously, the system's judgment on icing or slippery conditions is more comprehensive; when heavy rain, water depth, and abnormal vehicle acceleration occur simultaneously, the system's judgment on water accumulation risk is more reliable.

[0034] The fusion outputs a structured pavement condition data table, transforming data from different sources and formats into a unified data object that can be used for model recognition, early warning rule judgment, and management display. This feature prevents the fusion results from remaining at the level of raw sensor data, enabling subsequent pavement condition classification, hazard event generation, heat map display, and historical statistical analysis to be carried out based on a unified data structure.

[0035] Deep learning models receive multimodal fusion data and output road surface condition categories such as dry, wet, waterlogged, slippery, icy, and bumpy, along with confidence scores, enabling the system to form clear road surface condition judgments based on the fusion data. Compared to methods that judge a single parameter based on a threshold, multimodal models can comprehensively consider the correlation between vehicle dynamics, roadside environment, and meteorological conditions to classify and identify the complex differences between different road surface conditions.

[0036] A ResNet backbone network is used to extract temporal features from sensors, and a Transformer attention mechanism is combined to model the correlation between different data sources. This enables the model to extract local features formed by the changes in vehicle dynamics and environmental parameters over time, and to analyze the mutual influence between different data sources. This feature allows road condition recognition to go beyond simply superimposing multiple detection values; it enables the establishment of correlation judgments between multi-source heterogeneous data, improving the ability to identify complex road conditions.

[0037] The AI ​​model learns online based on feedback signals from the vehicle, enabling the system to continuously correct model parameters using feedback such as ABS trigger frequency and ESP intervention frequency generated during actual operation. Since vehicle dynamics responses vary across different cities, vehicle models, and road structures, the online learning mechanism allows the model to gradually adapt to the actual deployment environment, reducing the problem of deviations from real-world road scenarios after long-term operation of a fixed model.

[0038] The early warning rule engine generates slippage hazard events, water accumulation hazard events, and maintenance suggestion events based on the fusion analysis results, enabling road condition identification results to be transformed into actionable early warning or management objects. Compared to solutions that only provide condition classification results, the early warning rules map judgment conditions such as friction coefficient, water depth, and frequency of bump events to specific event types, facilitating collaborative use by driver-side early warning, management-side handling, and maintenance-side decision-making.

[0039] The system filters warning information based on the driver's current route, time, and location, establishing a correlation between the warning content and the risky road sections the vehicle is about to pass through. Compared to broadcast warnings, this feature reduces distracting information unrelated to the driver's current route, allowing the driver to receive alerts that more closely reflect actual driving risks, thereby improving the relevance and usability of the warning information.

[0040] Multimodal interaction methods, such as pop-up windows and voice prompts, enable drivers to receive timely warnings of road hazards while driving. Since it is not suitable for drivers to look at complex interfaces for extended periods while driving, pop-up windows and voice prompts can convey road condition risks to drivers in a more direct way, adapting to road scenarios that require rapid response, such as emergency flooding, icing, and slippery conditions.

[0041] This feature allows traffic management personnel to view heat maps showing the dry and wet conditions of the road network, event distribution, and trend statistics, enabling city-level road condition perception results to be presented in a regional context. Compared to single-point alarm information, this feature helps management observe risk distribution, changing trends, and high-incidence road sections at the city road network level, providing a data foundation for traffic control, temporary warning settings, and resource allocation.

[0042] Based on accumulated road surface condition data, preventative maintenance recommendations are generated, enabling the system to transform repeated water accumulation, bumps, or slippery events into road maintenance clues. Compared to relying on manual inspections or post-accident handling, continuously accumulated road surface condition data can reflect long-term drainage, pothole, or adhesion risks on a particular road section, providing a basis for maintenance departments to determine inspection priorities.

[0043] Redis caches real-time data and sets up a scrolling window, enabling the system to save recent vehicle, roadside, and meteorological data for real-time fusion, event tracking, and short-term trend analysis. Since road condition changes exhibit significant temporal continuity, real-time caching allows the system to compare current data with recent conditions within a certain timeframe, avoiding isolated judgments based on single sampling results.

[0044] The persistent storage of historical events, statistical data, and system configurations using a MySQL relational database allows for the long-term accumulation of road condition perception results, which can then be used for statistical analysis, rule configuration, and maintenance decisions. Compared to solutions that only provide real-time warnings, historical data storage enables the tracking of frequently flooded road sections, frequently bumpy road sections, and areas with long-term slippery risks, providing continuous data support for urban road management.

[0045] By employing low-latency processing methods such as MQTT long-connection communication, localized deployment of edge computing nodes, and message compression, the response time of the link between vehicle-side data collection, cloud-based fusion analysis, early warning generation, and early warning issuance is controlled. This feature targets application scenarios requiring rapid alerts for road hazards, compressing the communication and computational latency that may arise from multi-source fusion systems, enabling risks such as water accumulation, icing, or slippery conditions ahead to be communicated to following vehicles more promptly. Attached Figure Description

[0046] Figure 1 System block diagram; Figure 2 This is a flowchart. Detailed Implementation

[0047] To make the advantages and benefits of the technical solution provided by the present invention clearer, the technical solution provided by the present invention will now be described in further detail with reference to the accompanying drawings, specifically: Implementation Method 1: This implementation method provides a city-level road condition perception and early warning method based on vehicle-road-cloud multi-source data fusion, including: The steps involve collecting vehicle dynamics data, vehicle position data, and time information generated by the vehicle during its journey on the road, as well as collecting roadside environmental data and meteorological data, to form multi-source road condition perception data. The step of matching the multi-source road state perception data to the corresponding road unit and the corresponding time window based on vehicle location data, roadside environment data, road area corresponding to meteorological data, and time information to form multi-source matching data for road units. Based on the vehicle dynamics data in the multi-source matching data of the road unit, extract vehicle dynamics indirect perception indicators to characterize changes in road surface conditions. The vehicle dynamics indirect perception indicators include at least the step of indirect perception information formed by wheel speed differences, acceleration anomalies, braking state changes, and steering response changes. The steps involve taking vehicle dynamics indirect perception indicators, roadside environmental data, and meteorological data within the same road unit and the same time window as fusion objects, and determining the fusion weight of each fusion object based on the reliability of data sources, the consistency of multi-vehicle indirect perception information, and the correspondence between meteorological data and roadside environmental data and vehicle dynamics indirect perception indicators. The steps are as follows: weighted fusion of vehicle dynamics indirect perception indicators, roadside environmental data and meteorological data according to the fusion weights to form structured pavement state data for the corresponding road units; The steps for identifying the road condition category and condition confidence level of the corresponding road unit based on the structured road condition data, and forming the road condition identification result; The steps are as follows: generating road condition events based on the road condition recognition results, matching the road condition events with the vehicle's current location and driving route, and forming driver warning information related to the vehicle's driving route.

[0048] The vehicle dynamics data includes vehicle speed, longitudinal acceleration, lateral acceleration, wheel speeds, braking pressure, and steering angle. The roadside environmental data includes road surface temperature, water depth, icing status, and coefficient of friction. The meteorological data includes precipitation, air temperature, dew point temperature, and visibility.

[0049] When matching the multi-source road condition perception data to the corresponding road unit and the corresponding time window, the road unit corresponding to the vehicle dynamics data is determined based on the vehicle position data, the road unit corresponding to the roadside environment data is determined based on the road position corresponding to the roadside environment data, and the road unit covered by the meteorological data is determined based on the road area corresponding to the meteorological data. The vehicle dynamics data, roadside environment data and meteorological data in the same time window within the same road unit are combined to form the multi-source matching data of the road unit.

[0050] The wheel speed difference is determined by the deviation relationship between the wheel speeds of each wheel; the acceleration anomaly is determined by the deviation relationship between longitudinal acceleration and lateral acceleration relative to the historical normal driving state; the braking state change is determined by the brake pressure fluctuation and ABS trigger state; and the steering response change is determined by the correspondence between the steering angle and lateral acceleration.

[0051] When determining the fusion weights of each fusion object, the fusion weight of roadside environmental data is increased when valid roadside environmental data exists in the same road unit; the fusion weight of vehicle dynamics indirect perception indicators of multiple vehicles in the same road unit is increased when they have a consistent abnormal direction; and the fusion confidence of the corresponding road unit is increased when meteorological data, roadside environmental data, and vehicle dynamics indirect perception indicators have a state correspondence.

[0052] The structured pavement condition data includes road unit identifiers, time windows, vehicle dynamics fusion features, roadside environment fusion features, meteorological fusion features, and comprehensive confidence scores, and serves as the data basis for identifying pavement condition categories and condition confidence scores.

[0053] A city-level road condition perception and early warning device based on vehicle-road-cloud multi-source data fusion includes: The module collects vehicle dynamics data, vehicle position data, and time information generated by the vehicle during its journey on the road, and also collects roadside environmental data and meteorological data to form multi-source road condition perception data. Based on vehicle location data, roadside environment data corresponding to road location, meteorological data corresponding to road area, and time information, the multi-source road state perception data is matched to the corresponding road unit and the corresponding time window to form a module for multi-source matching data of road unit. Based on the vehicle dynamics data in the multi-source matching data of the road unit, vehicle dynamics indirect perception indicators for characterizing changes in road surface condition are extracted. The vehicle dynamics indirect perception indicators include at least a module of indirect perception information formed by wheel speed difference, acceleration anomaly, braking state change and steering response change. The module uses vehicle dynamics indirect perception indicators, roadside environmental data, and meteorological data within the same road unit and the same time window as fusion objects, and determines the fusion weight of each fusion object based on the reliability of data sources, the consistency of multi-vehicle indirect perception information, and the correspondence between meteorological data and roadside environmental data and vehicle dynamics indirect perception indicators. According to the fusion weight, the indirect perception indicators of vehicle dynamics, roadside environmental data and meteorological data are weighted and fused to form a module of structured pavement state data for the corresponding road unit; Based on the structured pavement condition data, a module is formed to identify the pavement condition category and condition confidence level of the corresponding road unit, thus generating a pavement condition identification result. Based on the road surface condition recognition results, a road condition event is generated, and the road condition event is matched with the vehicle's current location and driving route to form a module for driver warning information related to the vehicle's driving route.

[0054] A computer storage medium for storing a computer program, which, when read by the computer, is executed by the computer using the method described thereon.

[0055] A computer, including a processor and a storage medium, executes the method when the processor reads a computer program stored in the storage medium.

[0056] A computer program product, which, as a computer program, implements the method when the computer program is executed.

[0057] Implementation Method Two: This implementation method is a further detailed description of the technical solution provided in Implementation Method One, specifically: In one implementation, a city-level road condition perception and early warning method based on vehicle-road-cloud multi-source data fusion is used to perceive the physical conditions of urban roads, such as dryness, humidity, water accumulation, slipperiness, icing, and bumpiness, and output corresponding early warning or management information to drivers, traffic management personnel, or road maintenance departments based on the perception results. This method uses vehicle dynamics data, roadside environmental data, and meteorological data as multi-source inputs, and forms a continuous processing flow through unified access, spatiotemporal alignment, indirect perception indicator extraction, confidence-weighted fusion, road condition identification, and early warning distribution.

[0058] The system processes vehicle dynamics and position data generated by the vehicle during operation. The vehicle acquires vehicle speed, longitudinal acceleration, lateral acceleration, wheel speeds, braking pressure, and steering angle via the vehicle bus interface, and simultaneously acquires vehicle position data and the corresponding acquisition time. Vehicle dynamics data reflects the actual contact response between the vehicle and the road surface, vehicle position data determines the corresponding road location, and the acquisition time is used for subsequent time matching with roadside environmental and meteorological data. The raw vehicle data is encapsulated into a data record containing vehicle identification, acquisition time, road location, and vehicle dynamics parameters, and uploaded to the cloud data receiving layer via the vehicle communication link, serving as input for subsequent extraction of indirect vehicle dynamics perception indicators.

[0059] Processing of roadside environmental and meteorological data. Roadside sensing devices collect data on road surface temperature, water depth, icing status, and friction coefficient at the corresponding road location. Meteorological data sources collect or provide precipitation, air temperature, dew point temperature, and visibility. Roadside environmental data provides direct information on the physical condition of the road surface at a specific road location, while meteorological data provides environmental background information related to changes in road surface condition within a region. Both roadside environmental and meteorological data carry the collection or update time and corresponding spatial range, and are uploaded to the cloud data receiving layer, forming multi-source road condition sensing data together with the raw data from the vehicle end.

[0060] The system performs unified access and validity verification of multi-source road condition perception data. After receiving data from vehicle-side, roadside, and meteorological data sources, the cloud-based data receiving layer first parses the vehicle dynamics parameters, vehicle location data, roadside environmental parameters, and meteorological parameters according to a preset data format. Then, it determines whether each data record contains missing fields, abnormal timestamps, invalid locations, or parameter values ​​that significantly exceed reasonable ranges. For data records that pass verification, they are converted into a unified data structure and forwarded to the data processing layer; for data records that fail verification, they are marked as abnormal data or filtered, ensuring that the data entering subsequent fusion processes has a unified format and basic usability.

[0061] Spatial alignment is performed on the verified multi-source road condition sensing data. The data processing layer unifies the corresponding locations of vehicle location data, roadside environment data, and meteorological data under the same spatial reference. Based on road network information, it maps vehicle data to corresponding road units, roadside environment data to the road unit where its detection point is located, and meteorological data to road units within its coverage area. For vehicle data, the road unit where the vehicle is located at the time of collection is determined based on the vehicle location data; for roadside environment data, the corresponding road unit or adjacent road unit is determined based on the deployment location of roadside equipment; for meteorological data, the road area that can participate in the judgment is determined based on the coverage area of ​​the meteorological data. After spatial alignment, data from different sources can form a fusionable dataset around the same road unit.

[0062] The spatially aligned multi-source road condition perception data undergoes temporal alignment and sampling difference compensation. The data processing layer normalizes the timestamps of vehicle-side data, roadside environmental data, and meteorological data using a unified time base, and matches data within similar time ranges within the same road unit according to the time window required for road condition judgment. For data sources with different sampling frequencies, interpolation is performed based on adjacent sample values ​​or the most recent valid value is retained, ensuring that high-frequency vehicle dynamics data, low-frequency roadside environmental data, and periodic meteorological data can form a correspondence within the same time window. The time-aligned dataset is used for subsequent indirect perception indicator extraction and confidence-weighted fusion, avoiding incorrect matching of the same road condition due to inconsistent update times from different data sources.

[0063] The time-aligned vehicle-side data undergoes denoising and outlier removal. The data processing layer filters vehicle dynamics parameters to reduce the impact of sensor noise, communication jitter, or transient anomalous sampling on subsequent judgments, and identifies data that significantly deviates from the normal range of similar vehicles, the same road segment, or adjacent time windows as outliers. For outliers, removal, weight reduction, or retention can be selected based on the degree of anomaly. This preprocessed vehicle dynamics data serves as the basis for calculating indirect vehicle dynamics perception indicators, making the indicators extracted from subsequent vehicle responses more closely approximate the true dynamic characteristics caused by changes in road conditions.

[0064] The process involves extracting indirect vehicle dynamics perception indicators from the preprocessed vehicle dynamics data. The data processing layer extracts wheel speed difference based on the differences between wheel speeds, reflecting whether abnormal slippage occurs during vehicle operation; it extracts acceleration anomaly based on the deviation of current longitudinal and lateral acceleration from historical normal or adjacent time states, reflecting whether abnormal fluctuations occur in vehicle motion response; it extracts braking characteristic parameters based on brake pressure changes and ABS trigger status, reflecting whether low adhesion exists during vehicle braking; and it extracts steering response features based on the correspondence between steering angle and lateral acceleration, reflecting whether the vehicle's lateral response weakens under the same or similar steering inputs. These indirect vehicle dynamics perception indicators are associated and stored with corresponding road units, acquisition time, and vehicle identification, and serve as vehicle-side inputs in subsequent multi-source fusion.

[0065] The data processing layer performs aggregation and judgment on indirect vehicle dynamics perception indicators within the same road unit. For indirect vehicle dynamics perception indicators from one or more vehicles within the same road unit and the same time window, the data processing layer aggregates them according to the number of vehicles, vehicle passage time, direction of indicator anomaly, and degree of indicator anomaly. When only single-vehicle data is available, the indirect vehicle dynamics perception indicators of that vehicle are used as candidate indirect state information for that road unit. When multiple vehicles exhibit the same or similar wheel speed anomalies, acceleration anomalies, braking anomalies, or steering response anomalies at similar locations and within similar time periods, the indirect vehicle dynamics state information of that road unit is marked as consistently abnormal. This aggregation result enters the subsequent confidence-weighted fusion process to determine whether the vehicle-side data can provide common evidence of the road state.

[0066] The data processing layer constructs fusion input data by fusing vehicle dynamics indirect state information, roadside environmental data, and meteorological data within the same road unit. Taking road units as objects, the data processing layer organizes the aggregated vehicle dynamics indirect state information, the corresponding roadside environmental data, and the meteorological data covering that road unit into fusion input data. The fusion input data includes vehicle dynamics, roadside environmental, and meteorological dimensions. The vehicle dynamics dimension includes at least wheel speed difference, acceleration anomaly, braking characteristic parameters, and steering response characteristics; the roadside environmental dimension includes at least pavement temperature, water depth, icing status, and friction coefficient; and the meteorological dimension includes at least precipitation, air temperature, dew point temperature, and visibility. The fusion input data serves as the basis for dynamic confidence-weighted processing.

[0067] The data processing layer assigns initial confidence levels to the fused input data based on the reliability of the data source. The initial confidence levels are assigned according to the source type of vehicle dynamics indirect state information, roadside environmental data, and meteorological data. Roadside environmental data reflects the direct physical state of the road surface at fixed detection points and is assigned a higher initial confidence level when the data is valid and has a clear correspondence with road units. Vehicle dynamics indirect state information originates from the actual driving response of vehicles but may be affected by driving behavior, vehicle type differences, or vehicle state, and is assigned a medium initial confidence level. Meteorological data typically reflects regional environmental conditions and has relatively low spatial resolution, so it is assigned a relatively low initial confidence level. These initial confidence levels are used for subsequent dynamic adjustments based on data consistency and environmental conditions.

[0068] The processing layer adjusts the fusion weights based on the validity of roadside environmental data within a road unit. When valid roadside environmental data exists in a road unit, the data processing layer increases the weight of roadside environmental data in the fusion process of that road unit, and uses pavement temperature, water depth, icing status, or friction coefficient as direct judgment criteria in the fusion process. When valid roadside environmental data is unavailable in a road unit, or when roadside environmental data is unavailable due to equipment malfunction, data loss, or excessively long update time, the data processing layer reduces the weight of roadside environmental data and correspondingly increases the reference proportion of vehicle dynamics indirect state information and meteorological data in the fusion process. This processing allows key road sections with direct detection values ​​to prioritize the use of measured information, while road sections without roadside equipment can still rely on vehicle and meteorological data for state perception.

[0069] The processing layer adjusts the confidence level of indirect vehicle dynamics state information based on multi-vehicle consistency. When only a single vehicle exhibits dynamic anomalies within the same road unit, the data processing layer treats this anomaly as a candidate anomaly and assesses its reliability by combining the vehicle's driving state with adjacent time data. When multiple vehicles within the same road unit exhibit consistent dynamic anomalies within a short timeframe, the data processing layer increases the confidence level of the indirect vehicle dynamics state information. When there are significant differences between multi-vehicle data, or when only a few vehicles exhibit anomalies while others do not, the data processing layer reduces the impact of these anomalies on the fusion results. This processing ensures that occasional anomalies caused by single-vehicle driving operations do not directly determine road conditions, while simultaneously enabling cross-validation of road unit states using a large number of vehicles.

[0070] The overall confidence level is adjusted based on the consistency between meteorological conditions, indirect vehicle dynamics information, and roadside environmental data. The data processing layer determines whether the environmental conditions reflected in the meteorological data, such as precipitation, low temperature, dew point, and visibility, correspond to the vehicle dynamics anomalies and roadside environmental detection results. When low temperature and dew point conditions match with road surface temperature, abnormal vehicle braking, or abnormal wheel speed, the overall confidence level for icing or slippery conditions is increased; when precipitation matches with water depth, abnormal vehicle acceleration, or abnormal braking, the overall confidence level for water accumulation or slippery conditions is increased; when meteorological conditions are inconsistent with vehicle dynamics anomalies or roadside environmental data, the weight of anomaly judgments derived from a single data source is reduced. The adjusted confidence level is used to form the fused structured road surface condition data.

[0071] The process involves generating structured pavement state data. The data processing layer, based on dynamically adjusted fusion weights, weights and fuses indirect vehicle dynamics state information, roadside environmental data, and meteorological data to generate structured pavement state data corresponding to road units. This structured pavement state data includes at least the road unit identifier, time window, vehicle dynamics fusion features, roadside environmental fusion features, meteorological fusion features, confidence scores for each data source, and a comprehensive confidence score. This structured pavement state data serves as input to the subsequent pavement state recognition model and also as the data foundation for rule-based judgments by the early warning rule engine.

[0072] Road condition identification is performed based on structured road condition data. The structured road condition data is input into the road condition identification model. Based on the correlation between vehicle dynamics fusion features, roadside environment fusion features, and meteorological fusion features, the model outputs the road condition category and corresponding confidence score for each road unit. Road condition categories include dry, wet, waterlogged, slippery, icy, and bumpy. The road condition category and confidence score output by the model together constitute the road condition identification result and are transmitted to the early warning rule engine. Optionally, the road condition identification model can employ a deep learning model capable of processing multimodal data and update the model parameters through vehicle-side feedback signals; however, this model structure is not limited to the only implementation method.

[0073] The processing of road condition events is based on the road surface condition identification results. After receiving the road surface condition identification results, the early warning rule engine determines whether to generate a road condition event based on the road surface condition category, corresponding confidence level, and relevant fusion features. When the road surface condition identification results indicate that a road unit has a risk of slipperiness or icing, and the comprehensive confidence level meets the preset conditions, a slipperiness hazard event is generated; when water depth, precipitation conditions, and abnormal vehicle dynamics all point to water accumulation, and the identification confidence level meets the preset conditions, a water accumulation hazard event is generated; when the same road unit continuously exhibits bumpy vehicle dynamics anomalies within a certain period of time, a bumpy road event or maintenance suggestion event is generated. A road condition event includes at least the event type, the event road unit, the event time, the risk level, and the confidence level, and serves as the input for display on the early warning distribution and management end.

[0074] Driver warnings are matched based on road condition events and vehicle routes. Upon receiving a road condition event, the warning service layer obtains the vehicle's current location and route, matching the road unit where the event occurs with the road unit the vehicle is about to pass through. If the road condition event is within the relevant range of the vehicle's current route, and the event time and risk level meet the warning conditions, the event is identified as a warning event relevant to the vehicle. If the road condition event is unrelated to the vehicle's current route, or if the event has exceeded its valid time frame, the warning event is not sent to the vehicle. Warning events filtered by route, time, and location are converted into driver warning information and pushed to the in-vehicle terminal.

[0075] The system processes multimodal warning information for the driver. After receiving the warning information from the driver, the onboard terminal generates corresponding display content or voice prompts based on the event type and risk level. For slippery road hazards, it can warn of the risk of slippery or icy conditions ahead; for water accumulation hazards, it can warn of the risk of water accumulation ahead; and for bumpy road hazards, it can warn of the upcoming bumpy road section. Driver warning information can be output via pop-up windows, voice prompts, or a combination of both on the onboard terminal, ensuring that road condition events are communicated to the driver before the vehicle approaches the corresponding road unit.

[0076] The processing of data for traffic management terminals is based on road condition events. The early warning service layer summarizes the road surface condition identification results and road condition events of each road unit according to road location, event type, time range, and risk level to generate road network condition display data. The traffic management terminal displays the road dry / wet status, risk event distribution, and trend statistics based on the road network condition display data, and supports viewing by road area, event type, or time range. This processing enables traffic management personnel to grasp the regional road status based on the fused city-level road condition data, rather than relying solely on alarms from single detection points.

[0077] The process generates maintenance decision support information based on historical road condition events. The data storage layer continuously saves road condition events, pavement condition identification results, and related statistical data. The early warning service layer or management terminal uses historical data to statistically analyze the frequency of water accumulation, bumpy, or slippery events occurring within the same road unit over a certain time period. When a road unit repeatedly experiences similar road condition events within a preset time range, it is marked as a key focus road unit, and corresponding maintenance recommendations are generated. These recommendations may include the road unit location, event type, number of triggers, time distribution of occurrence, and suggested investigation targets, providing a basis for drainage facility inspections, pavement repairs, or temporary warning setups.

[0078] The system employs a tiered storage approach for real-time and historical data. Recent data from vehicle-side, roadside, and meteorological data, along with current road condition events, are stored in a real-time cache in the cloud to support continuous road condition assessment, short-term trend analysis, and repeated verification of early warnings. Historical road condition events, statistical data, and system configurations are persistently stored to support subsequent queries, statistical analysis, and maintenance decisions. Data in the real-time cache is updated according to a rolling time window; data exceeding the time window is either transferred to historical storage or cleared. Through this tiered management of real-time and historical data, current road condition assessment and long-term road management analysis can be seamlessly integrated within the same workflow.

[0079] In one optional implementation, delay constraints can be set for vehicle-side data collection, cloud reception, fusion computing, early warning generation, and early warning distribution, and long-lived communication, proximity computing nodes, or message compression methods can be used to reduce data transmission and processing waiting time. Data uploaded from the vehicle side, after passing validity checks, is prioritized for entry into the real-time processing queue. The early warning service layer prioritizes route matching and vehicle-side distribution for high-risk road condition events. This optional approach is designed to adapt to road scenarios requiring timely alerts, such as water accumulation, slippery conditions, or icy conditions, but it does not alter the main process of fusing vehicle dynamics data, roadside environmental data, and meteorological data to identify road surface conditions.

[0080] In one optional implementation, the road condition recognition model can receive vehicle-side feedback data for updates. This feedback data includes ABS trigger status, ESP intervention status, or the vehicle's dynamic response after the driver passes through a warning section. The cloud-based system associates the feedback data with previously generated road condition recognition results. When the feedback data matches the recognition results, the reliability of the corresponding sample is increased; when there are discrepancies, the sample can be marked as data to be corrected and used for subsequent model parameter updates or rule adjustments. This optional approach allows the model to adapt to data differences across different road areas, vehicle types, and environmental conditions.

[0081] In one optional implementation, when a data source is temporarily unavailable, the cloud can continue to generate road condition recognition results based on available data sources, and record the missing data source and corresponding confidence level adjustments in the structured road surface condition data. When roadside environmental data is missing, vehicle dynamics indirect state information and meteorological data are fused; when vehicle-side data is insufficient, roadside environmental data and meteorological data are fused; when meteorological data is delayed in updating, vehicle dynamics indirect state information and roadside environmental data are fused. This processing enables the method to maintain continuous operation under different deployment conditions, while the confidence level record reflects the reliability of the recognition results.

[0082] Implementation Method 3, in conjunction with Appendix Figure 1-2 This embodiment describes the technical solution provided above in further detail through specific examples. Specifically: Vehicle-side perception layer: Deployed in connected vehicles (such as large-scale taxi operations), it collects vehicle dynamic parameters in real time via the OBD-II / CAN bus interface, including at least 10 parameters such as vehicle speed, longitudinal acceleration, lateral acceleration, wheel speeds, braking pressure, and steering angle, with a sampling frequency of no less than 10Hz. It combines GPS / BeiDou dual-mode positioning modules to obtain vehicle location information (accuracy better than ±1 meter), and uploads the collected data to the cloud via an onboard 4G / 5G communication module using HTTP or MQTT protocols.

[0083] Roadside sensing layer: Deployed at key intersections or road sections in the city, it includes road surface condition sensors (detecting road surface temperature, water depth, icing status, friction coefficient, etc.) and meteorological monitoring stations (collecting environmental parameters such as wind speed, precipitation, visibility, and PM2.5). Data is uploaded to the cloud via 4G / 5G or LoRaWAN networks. The roadside equipment is powered by a combination of solar energy and municipal power and supports IP67 protection rating.

[0084] Cloud Platform - Data Receiving Layer: The multi-protocol data access gateway is responsible for receiving multi-source data from vehicle terminals, roadside terminals, and third-party meteorological services. It supports multiple protocols such as HTTPS, WebSocket, MQTT, and TCP Socket, verifies the data format and integrity, filters abnormal data, and forwards valid data to the data processing layer.

[0085] Cloud Platform - Data Processing Layer: Spatiotemporal alignment and preprocessing module: adopts UTM coordinate system as a spatial reference, aligns timestamps to a unified benchmark (such as UTC+8), compensates for the sampling frequency differences of different sensors through interpolation algorithm; performs noise reduction processing on sensor data based on Kalman filtering, and removes outliers based on the 3σ principle.

[0086] Multi-source data fusion engine: This is the core module of this invention. Based on a confidence-weighted strategy, fusion weights are dynamically allocated according to the reliability of each data source. Fusion dimensions include: (a) vehicle dynamics characteristics (acceleration anomalies, wheel speed differences, ABS trigger frequency, braking pressure fluctuations); (b) roadside environmental parameters (road surface temperature, water depth, measured friction coefficient); (c) meteorological data (precipitation, air temperature, dew point temperature, visibility). The fusion output is a structured road surface condition data table.

[0087] AI road condition recognition model: Employing a deep learning model (such as a ResNet backbone network combined with a Transformer attention mechanism), it takes multimodal fusion data as input and outputs road condition classifications (dry / wet / waterlogged / slippery / icy / bumpy) and corresponding confidence scores. The model supports an online learning mechanism, continuously optimizing parameters based on vehicle feedback (such as ABS trigger frequency and ESP intervention frequency).

[0088] Cloud Platform - Early Warning Service Layer: Early warning rule engine: Triggers early warning rules based on fusion analysis results. For example: a slippage hazard event is generated when the road surface friction coefficient is lower than a set threshold; a water accumulation hazard event is generated when the water depth exceeds a set threshold; a maintenance suggestion event is generated when the frequency of bump events on a certain road section exceeds a set threshold.

[0089] Personalized filtering and distribution module: For drivers, it filters warning information based on current driving route, time and location, and provides multimodal interaction (including in-vehicle terminal pop-ups and voice prompts); for traffic management personnel, it visualizes the road network's dry and wet status heat map, event distribution and trend statistics on a large screen, and supports data drill-down and multi-dimensional analysis.

[0090] Data storage layer: Redis is used to cache real-time data (24-hour rolling window), and MySQL relational database is used to persistently store historical events, statistical data and system configurations, supporting distributed deployment and master-slave replication.

[0091] In application: Step 1: Extraction of indirect perception indicators for vehicle dynamics The following dynamic indicators that indirectly reflect road conditions are extracted from the vehicle's CAN bus data: Wheel speed difference: The wheel slip ratio is calculated by comparing the readings of each wheel speed sensor. When the road surface is slippery, the wheel speed difference between the drive wheel and the driven wheel increases significantly.

[0092] Acceleration Anomaly: Compares the current longitudinal / lateral acceleration with historical normal values. When a vehicle is traveling on a road surface with a low coefficient of friction, even with gentle driver input, the acceleration response may exhibit abnormal fluctuations.

[0093] Braking characteristic parameters: When the vehicle triggers ABS (Anti-lock Braking System), the frequency of ABS triggering and the amplitude of brake pressure fluctuations are recorded. A high ABS triggering frequency is a strong indication of a wet / icy road surface.

[0094] Steering response characteristics: Monitor the response relationship between steering angle and lateral acceleration. When the road adhesion coefficient decreases, the lateral acceleration response generated by the same steering angle weakens.

[0095] Step 2: Confidence-weighted spatiotemporal fusion Assign dynamic confidence weights to each data source:

[0096] The fusion weights are dynamically adjusted according to the following rules: When a road section has direct measured values ​​from roadside sensors, its weight is given priority. When multiple vehicles generate consistent dynamic anomaly signals on the same road segment, the confidence level of the indirect indicators for that road segment increases. When meteorological conditions (such as heavy rain and low temperature) are consistent with abnormal vehicle dynamics signals and roadside sensor readings, the overall confidence level is further improved.

[0097] Step 3: AI Model Multimodal Fusion Inference The structured fusion data output from step two is input into a deep learning model. The model uses a ResNet backbone network to extract temporal features from the sensors and combines the Transformer self-attention mechanism to model the correlation between different data sources. Finally, the road surface state category and confidence level are output through the classification head.

[0098] Key technical parameters: End-to-end warning latency: The end-to-end latency from vehicle data collection to warning display on the vehicle terminal does not exceed 500ms.

[0099] Vehicle positioning accuracy: GPS / BeiDou dual-mode positioning, with an accuracy better than ±1 meter (95% confidence interval).

[0100] Vehicle-side data sampling frequency: CAN bus data sampling frequency not less than 10Hz, roadside sensor sampling frequency not less than 1Hz.

[0101] AI model inference speed: single road condition prediction takes no more than 50ms (GPU deployment).

[0102] Overall early warning accuracy: The accuracy rate for identifying dry / humid / waterlogged conditions is no less than 85%, and the accuracy rate for identifying icy conditions is no less than 80%.

[0103] Data caching window: Real-time data is stored in a 24-hour rolling window in Redis, while historical event data is persistently stored in MySQL.

[0104] System availability: The cloud platform SLA is no less than 99.9%, and it supports dual-node hot standby and automatic failover.

[0105] Concurrent access capability: Supports no less than 1,000 vehicles to be online simultaneously and upload data in real time.

[0106] Compared with existing technologies, the advantages of this implementation are: Comprehensive perception dimensions: This invention is the first to organically integrate vehicle CAN bus dynamics data, roadside environmental sensor data and meteorological data, realizing a technological leap from "single physical sensor" to "vehicle-road-cloud multi-dimensional collaborative perception".

[0107] Innovative use of vehicles as mobile sensors: This invention creatively uses the dynamic response of vehicles in large-scale operation as "indirect sensors" of road conditions, making each connected vehicle in motion a mobile node of the road condition perception network. This achieves city-level road condition coverage at extremely low marginal cost, overcoming the pain points of high deployment cost and large coverage blind spots of roadside fixed sensors.

[0108] Confidence-weighted dynamic fusion mechanism: The fusion strategy of this invention dynamically adjusts the fusion weight of each data source based on the reliability of the data source, the consistency verification of multiple vehicles, and the synergistic corroboration of environmental conditions. Compared with the fusion method of fixed weight or simple averaging, it significantly improves the robustness and accuracy of the perception results.

[0109] End-to-end low latency optimization: Through engineering methods such as layered latency budget allocation (vehicle-side data collection <80ms, cloud processing <50ms, early warning issuance <70ms), MQTT long connection communication, and local deployment of edge computing nodes, an end-to-end early warning response of less than 500ms is achieved, meeting the real-time requirements of active safety scenarios on urban roads.

[0110] Personalized tiered early warning distribution: Unlike the broadcast-style early warning of existing technologies, this invention provides drivers with precise early warnings filtered by route, time and location, traffic management personnel with an overview of the regional situation and statistical analysis, and maintenance departments with preventive maintenance suggestions driven by road condition data, realizing differentiated services for multiple roles on one platform.

[0111] Online learning and continuous evolution: The AI ​​model receives actual feedback signals from the vehicle (such as ABS trigger frequency and ESP intervention events), forming a data closed loop of "perception → warning → feedback → optimization", which enables the model to continuously optimize as the system runs.

[0112] In a specific embodiment: Example 1: Real-time early warning of slippery road surfaces at the city level Application scenario: The system of this invention has been deployed in a city, with 150 connected taxis operating as mobile sensing nodes, and roadside sensor devices deployed at 5 key intersections.

[0113] Work process: During operation, the connected taxi continuously collects CAN bus data (vehicle speed, wheel speed, acceleration, braking pressure, etc.) through the OBD-II interface at a sampling frequency of 10Hz, and uploads it to the cloud via the 4G network along with GPS positioning data.

[0114] The roadside sensors (road surface condition sensor + weather station) collect environmental data (road surface temperature -5℃, water depth 0mm, wind speed 3m / s) at 1-minute intervals and upload them synchronously.

[0115] Access third-party meteorological service API to obtain regional forecast data (air temperature -3℃, dew point temperature -5℃, precipitation 0mm).

[0116] The cloud-based fusion engine detected that multiple vehicles triggered ABS at similar locations on a certain road section (near the intersection of XX Road and XX Road), with wheel speed differences exceeding the historical average by more than three times the standard deviation, and increased brake pressure fluctuations. Simultaneously, roadside surface temperature sensors at the same location showed a surface temperature of -2℃.

[0117] AI model fusion reasoning: abnormal vehicle dynamics signal + low roadside temperature + meteorological dew point temperature lower than road surface temperature → overall confidence level 0.89, judged as "icy / slippery" state.

[0118] The warning engine generates a "slippery danger" event, and the personalized distribution module matches the vehicles that are about to pass through the section of road, pushing the warning information to the vehicle terminal: "The road surface may be icy about 200 meters ahead. It is recommended to slow down."

[0119] Meanwhile, the section of road is marked as a high-risk area (red) on the large screen using a heat map, which traffic management personnel can use to dispatch de-icing vehicles or set up temporary warnings.

[0120] Technical results: From the first vehicle detecting the abnormal dynamic signal to the warning being pushed to subsequent vehicles, the end-to-end delay was 387ms; on the same day, the system helped avoid at least 3 traffic accidents caused by black ice on the road.

[0121] Example 2: Road Flooding Early Warning and Maintenance Decision Support Application scenario: During heavy rain, some sections of the city's drainage system experience water accumulation.

[0122] Work process: A roadside water accumulation sensor detected a water depth of 5mm on a certain road section (exceeding the 3mm warning threshold) and uploaded the data to the cloud.

[0123] Multiple vehicles passing through this section of road showed consistent abnormal patterns in instantaneous deceleration and lateral acceleration in their CAN data, which are consistent with the characteristics of hydroplaning.

[0124] Meteorological data confirms that the hourly precipitation in the area reached 15 mm.

[0125] The fusion confidence level was increased to 0.92, and the early warning engine generated a "flooding hazard" event.

[0126] The system sends warnings to drivers and simultaneously generates a 24-hour water accumulation event distribution map and road section water accumulation frequency statistics on the large screen.

[0127] The system automatically generates a maintenance recommendation report, indicating that road sections that have triggered more than 10 waterlogging warnings in the past 30 days should have their drainage facilities inspected first.

[0128] Example 3: Identification and Preventive Maintenance of Bumpy Road Sections Application scenario: Potholes appear on some parts of urban roads, affecting driving comfort and safety.

[0129] Work process: When multiple vehicles passed through a certain road section, the longitudinal acceleration and vertical vibration data recorded by the CAN bus showed consistent abnormal peaks.

[0130] The system marks this road segment as a "bumpy road segment" and accumulates the frequency of bumpy events.

[0131] When the number of bumpy events on a certain road section exceeds a set threshold (e.g., 50 times) in a single day, a maintenance work order suggestion will be automatically generated.

[0132] Traffic management departments can view heat maps of bumpy road sections on a large screen and arrange maintenance and construction according to priority.

[0133] The above description of several specific embodiments further details the technical solution provided by the present invention in order to highlight the advantages and benefits of the technical solution provided by the present invention. However, the above-described specific embodiments are not intended to limit the present invention. Any reasonable modifications and improvements to the present invention, combinations of embodiments, and equivalent substitutions based on the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A city-level road condition perception and early warning method based on vehicle-road-cloud multi-source data fusion, characterized in that, include: The steps involve collecting vehicle dynamics data, vehicle position data, and time information generated by the vehicle during its journey on the road, as well as collecting roadside environmental data and meteorological data, to form multi-source road condition perception data. The step of matching the multi-source road state perception data to the corresponding road unit and the corresponding time window based on vehicle location data, roadside environment data, road area corresponding to meteorological data, and time information to form multi-source matching data for road units. Based on the vehicle dynamics data in the multi-source matching data of the road unit, extract vehicle dynamics indirect perception indicators to characterize changes in road surface conditions. The vehicle dynamics indirect perception indicators include at least the step of indirect perception information formed by wheel speed differences, acceleration anomalies, braking state changes, and steering response changes. The steps involve taking vehicle dynamics indirect perception indicators, roadside environmental data, and meteorological data within the same road unit and the same time window as fusion objects, and determining the fusion weight of each fusion object based on the reliability of data sources, the consistency of multi-vehicle indirect perception information, and the correspondence between meteorological data and roadside environmental data and vehicle dynamics indirect perception indicators. The steps are as follows: weighted fusion of vehicle dynamics indirect perception indicators, roadside environmental data and meteorological data according to the fusion weights to form structured pavement state data for the corresponding road units; The steps for identifying the road condition category and condition confidence level of the corresponding road unit based on the structured road condition data, and forming the road condition identification result; The steps are as follows: generating road condition events based on the road condition recognition results, matching the road condition events with the vehicle's current location and driving route, and forming driver warning information related to the vehicle's driving route.

2. The urban-level road condition perception and early warning method based on vehicle-road-cloud multi-source data fusion according to claim 1, characterized in that, The vehicle dynamics data includes vehicle speed, longitudinal acceleration, lateral acceleration, wheel speeds, braking pressure, and steering angle. The roadside environmental data includes road surface temperature, water depth, icing status, and coefficient of friction. The meteorological data includes precipitation, air temperature, dew point temperature, and visibility.

3. The urban-level road condition perception and early warning method based on vehicle-road-cloud multi-source data fusion according to claim 1, characterized in that, When matching the multi-source road condition perception data to the corresponding road unit and the corresponding time window, the road unit corresponding to the vehicle dynamics data is determined based on the vehicle position data, the road unit corresponding to the roadside environment data is determined based on the road position corresponding to the roadside environment data, and the road unit covered by the meteorological data is determined based on the road area corresponding to the meteorological data. The vehicle dynamics data, roadside environment data and meteorological data in the same time window within the same road unit are combined to form the multi-source matching data of the road unit.

4. The urban-level road condition perception and early warning method based on vehicle-road-cloud multi-source data fusion according to claim 1, characterized in that, The wheel speed difference is determined by the deviation relationship between the wheel speeds of each wheel; the acceleration anomaly is determined by the deviation relationship between longitudinal acceleration and lateral acceleration relative to the historical normal driving state; the braking state change is determined by the brake pressure fluctuation and ABS trigger state; and the steering response change is determined by the correspondence between the steering angle and lateral acceleration.

5. The urban-level road condition perception and early warning method based on vehicle-road-cloud multi-source data fusion according to claim 1, characterized in that, When determining the fusion weights of each fusion object, the fusion weight of roadside environmental data is increased when valid roadside environmental data exists in the same road unit; the fusion weight of vehicle dynamics indirect perception indicators of multiple vehicles in the same road unit is increased when they have a consistent abnormal direction; and the fusion confidence of the corresponding road unit is increased when meteorological data, roadside environmental data, and vehicle dynamics indirect perception indicators have a state correspondence.

6. The urban-level road condition perception and early warning method based on vehicle-road-cloud multi-source data fusion according to claim 1, characterized in that, The structured pavement condition data includes road unit identifiers, time windows, vehicle dynamics fusion features, roadside environment fusion features, meteorological fusion features, and comprehensive confidence scores, and serves as the data basis for identifying pavement condition categories and condition confidence scores.

7. A city-level road condition perception and early warning device based on vehicle-road-cloud multi-source data fusion, characterized in that, include: The module collects vehicle dynamics data, vehicle position data, and time information generated by the vehicle during its journey on the road, and also collects roadside environmental data and meteorological data to form multi-source road condition perception data. Based on vehicle location data, roadside environment data corresponding to road location, meteorological data corresponding to road area, and time information, the multi-source road state perception data is matched to the corresponding road unit and the corresponding time window to form a module for multi-source matching data of road unit. Based on the vehicle dynamics data in the multi-source matching data of the road unit, vehicle dynamics indirect perception indicators for characterizing changes in road surface condition are extracted. The vehicle dynamics indirect perception indicators include at least a module of indirect perception information formed by wheel speed difference, acceleration anomaly, braking state change and steering response change. The module uses vehicle dynamics indirect perception indicators, roadside environmental data, and meteorological data within the same road unit and the same time window as fusion objects, and determines the fusion weight of each fusion object based on the reliability of data sources, the consistency of multi-vehicle indirect perception information, and the correspondence between meteorological data and roadside environmental data and vehicle dynamics indirect perception indicators. According to the fusion weight, the indirect perception indicators of vehicle dynamics, roadside environmental data and meteorological data are weighted and fused to form a module of structured pavement state data for the corresponding road unit; Based on the structured pavement condition data, a module is formed to identify the pavement condition category and condition confidence level of the corresponding road unit, thus generating a pavement condition identification result. Based on the road surface condition recognition results, a road condition event is generated, and the road condition event is matched with the vehicle's current location and driving route to form a module for driver warning information related to the vehicle's driving route.

8. A computer storage medium for storing computer programs, characterized in that, When the computer program is read by the computer, the computer executes the method of claim 1.

9. A computer, comprising a processor and a storage medium, characterized in that, When the processor reads the computer program stored in the storage medium, the computer executes the method of claim 1.

10. A computer program product, as a computer program, is characterized by: When the computer program is executed, it implements the method of claim 1.

Images